From patchwork Tue May 23 13:47:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684934 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729982wrt; Tue, 23 May 2023 06:51:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ62CeMDk09YShUG8GYMpfwFNp1ykYeXnvet6iuGYBiKeUazGcWQ4/wnnd9qielMw9HYCemg X-Received: by 2002:ac8:5fc3:0:b0:3f6:a9bb:85be with SMTP id k3-20020ac85fc3000000b003f6a9bb85bemr10622356qta.67.1684849877880; Tue, 23 May 2023 06:51:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849877; cv=none; d=google.com; s=arc-20160816; b=rM7PQpiP1H5NkEu02o4l68OTF7IdhyduSDFYAuJHE1mZh8QwVE7E2Ofk5MogCr1fCI oC24Jmd3mcb2SCU3WUEabthqgrSGmLSYt14qjvz4WJSqAuHkRVY40h81/C5k9UgfetQy UaSXnk7mHvxDKcJUUSlePkUrhhFIm5eUZIs7pebhVWZUmdHW12usfZPMxl2ofTsVWTaS 5YE18//CWCYJLI4wziNfACp48AnVsBUrM+B11Bbs3t4GZMj5eS6Tmr3Yk7GhVbT3lI2A aSxjIcufgVfRKYAgfEmMVuLvbVmtqQwJVCavVIpYglvwqsVI9H1ADYHk2RtQdxfjC3aM uKKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=S2ONWtUa3lpwntXaI3kn7ynLd1Z1O2L+f7C0TvCO4wU=; b=A+LWmv6kLTE+46i8H49q5nQH0LSYroZfabmYx+jPnHvwEvk2AyxVWhPrg6+/NnjDGx tkZYBQtSU/YU8lSEou2ctZ+FQUoSMZWh3giUal/W8+yeU5yelhDf/hP5K0WV8O4/mNE/ UJO2Y5NduJkRo0QqOaCTuQsEjjHZty16LomV90NZ/bZvxZlMEh8xAjQyBZBcjQeCwHNh QezzZ6APcjdKhjOWmX3EW9nFOXyA9xElEhvOeyO/QePIgNJo1C+uXKBpx7ZoC2muogNG dDTS9wRnT1Qw+iqlkFe4StOJ+UJxA2tA3c/TvxWC1YBT3lFmL1JjHKsyNQWURxlxANUx avVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cqzGhOzm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t10-20020a05622a01ca00b003e38e551e6asi5298292qtw.358.2023.05.23.06.51.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:51:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cqzGhOzm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMW-0008A1-7U; Tue, 23 May 2023 09:47:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMV-00089S-5J for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:39 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMT-000136-J4 for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:38 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-64d2b42a8f9so4260844b3a.3 for ; Tue, 23 May 2023 06:47:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849656; x=1687441656; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=S2ONWtUa3lpwntXaI3kn7ynLd1Z1O2L+f7C0TvCO4wU=; b=cqzGhOzm1eS7GSolGIZ6egzVxNRnL09T2h/sjV+t8QzzkvNM7ZTjqMCRy1T4GjmM/z a7KXwAkl+f5ZKNcfLK9irey1pbEShZn1+i6RtnypD8saI65zwlcFFkthVUxS3w8dbJzN UgGm+b/f2w7MI82AJCE8g35fOqzH0DOcz1FZaNetBPKSJ3QW4/Ig7Cu2/4Dh6mt61/NT 2uuaTlkbB/SWHsJGGmjixYwdNjKxDoM1AcMKb9s97qgy2M5T5UK61OoQGSSEJdn+xmjs Gnd/NvHYD2Bw3cGjx8HMAIMJy9aCOb/lMxFkNRW37s9amzUd8VaX61UA69eYvqSUVXX6 ddFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849656; x=1687441656; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=S2ONWtUa3lpwntXaI3kn7ynLd1Z1O2L+f7C0TvCO4wU=; b=dAhkmUWFvfs7tgXqVwMtimQWro21vkNHNrnmSa/vQ6q8MYeOcd3U2VVtRc2pf+EJUg gsXQI+gVlAvzgGa/LlGNjv8u1oFfMa9XxbHJMtPPqZuntd9E509b0kOlLiQr/XUmxQAm +aE8XCbDJT3whzgWpRk72TEU36+wGfbaTtpDOPy5m7tlMDi33UwXaUVtstQOAby7l8sr OACUJFkZlwxt+gQqytzgwDQZ3dd1Ksd/rpISbCjTWP1EeccsfPgcPN5/jFAWzzmxA7T1 y/yf3ivlWPuwBR6srrQMsvs3uQ1h+eRZM4H/TlXlfmeKg/4PROTFaoN5nIqI1ZiotvQs TjuQ== X-Gm-Message-State: AC+VfDyiMVOdRBRmnK5P/fuDrUaEg/rfaEs6VnfL20RzX4TA39L9vJjJ Nsz9m0rEmK8K5kZXcu17cUyfWHpp2YxAd+F0Rck= X-Received: by 2002:a05:6a00:2d26:b0:643:b489:246d with SMTP id fa38-20020a056a002d2600b00643b489246dmr18078532pfb.3.1684849655941; Tue, 23 May 2023 06:47:35 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= , Juan Quintela Subject: [PATCH v2 01/27] util: Introduce host-specific cpuinfo.h Date: Tue, 23 May 2023 06:47:07 -0700 Message-Id: <20230523134733.678646-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The entire contents of the header is host-specific, but the existence of such a header is not, which could prevent some host specific ifdefs at the top of the file for the include. Add host/include/{arch,generic} to the project arguments. Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/generic/host/cpuinfo.h | 4 ++++ meson.build | 10 ++++++++++ 2 files changed, 14 insertions(+) create mode 100644 host/include/generic/host/cpuinfo.h diff --git a/host/include/generic/host/cpuinfo.h b/host/include/generic/host/cpuinfo.h new file mode 100644 index 0000000000..eca672064a --- /dev/null +++ b/host/include/generic/host/cpuinfo.h @@ -0,0 +1,4 @@ +/* + * No host specific cpu indentification. + * SPDX-License-Identifier: GPL-2.0-or-later + */ diff --git a/meson.build b/meson.build index 0a5cdefd4d..c516b911d9 100644 --- a/meson.build +++ b/meson.build @@ -512,6 +512,16 @@ add_project_arguments('-iquote', '.', '-iquote', meson.current_source_dir() / 'include', language: all_languages) +# If a host-specific include directory exists, list that first... +host_include = meson.current_source_dir() / 'host/include/' +if fs.is_dir(host_include / host_arch) + add_project_arguments('-iquote', host_include / host_arch, + language: all_languages) +endif +# ... followed by the generic fallback. +add_project_arguments('-iquote', host_include / 'generic', + language: all_languages) + sparse = find_program('cgcc', required: get_option('sparse')) if sparse.found() run_target('sparse', From patchwork Tue May 23 13:47:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684925 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729044wrt; Tue, 23 May 2023 06:49:18 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4nD0Khzgw+KfakQXgi54ysK6p6o4OiBFfNrJC/cOXbNKXGx+NCIMt6NQNJQZdoVv8J0mWZ X-Received: by 2002:a05:6214:c87:b0:623:b838:493f with SMTP id r7-20020a0562140c8700b00623b838493fmr23588975qvr.12.1684849758487; Tue, 23 May 2023 06:49:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849758; cv=none; d=google.com; s=arc-20160816; b=DxrSymyudyfIOVXtsVXlzcBmAWCWIzL871Gmy5Hr9AQ6y2+SVvqKHrkLoEsF6iQ9vl J4zsLt+AXa84GMKMFbcBBnStkAkraNXTnGOCFkGy/C0yEG+3RX509PgOsCC+Y0w2pTXp /azn8zXYI9AIRNXwfe5HWLViEjTsk0GTE6qMhbSrJn66TJDVl1LTkBDWNWD8a47ee1gt SVw2o4P3QY8uveZ6pBiswmnPs22xJoFup7JQrzf4Qdvmp4fvhUBC+4X8ycXnzMyosrk3 DwDSLz2Wku5pjrslE6alCd2MJdUaIrY2IZk1UH/AVmKk2Lpv88j4uT9UAeIDxzEXCpCM 8KsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=wlUqj841x578nzMKn5ImdhcaDhdsmsJg4wjQFvgryxE=; b=d0a/fvIQPiQoGIbZng2OYcIde0MfO5YX53FL+RWxFZjG8/n/kFAkBq0xlztyldfKDe oL/6Tj5m5mQ9kEd0w7M5TiqTTdeTTzV2P3F4UtaYZ3Te9zYoRm6QzpyB3Q3a9VePgp0w n3mzhvgQlZnDGo2VBOoCeS0TehquuvuzctRuAjMSvw8jZxgO4wnaZdzmClyqhTKc6Sjq 8e2jrrQwXeJ3JdCxI+bFI9vxFZiqBhcwr97nWYrGjSf1szopdpM7uJ3GIX/H7CoXZZOM SE18f5Ro4ax6HUPxL1cJXASwd0blr2dBPCajbh9uS/MGArTdcMEEK0jBWde5YfJ6U/Oc B5Vw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jPmxL2Cj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id fn3-20020ad45d63000000b0062590a90a2fsi1914549qvb.325.2023.05.23.06.49.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:49:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jPmxL2Cj; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMY-0008CZ-Od; Tue, 23 May 2023 09:47:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMW-0008AJ-Ff for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:40 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMU-00013F-EZ for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:40 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d24136685so3498885b3a.1 for ; Tue, 23 May 2023 06:47:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849657; x=1687441657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wlUqj841x578nzMKn5ImdhcaDhdsmsJg4wjQFvgryxE=; b=jPmxL2Cjx8jI4jVIUYCN8Vsk8sHHHfPgDz+G60sgjLPvCWUIh6+YlW3YaWXczb/S/7 Ue+Y2Hn9jFI4cos/6i30iMalHFz6KvNLgl/U/B2iwaDqokjpTIxJ/Vr1X59T+4A86kA2 XyjEA3yPEMs2Nw7g56mWZQC+/xxKtr67lKHCv+CMT538EA2q1C87lTu8D5PDCUd7XSN9 hH0hB3OaWeBHF6NLGaHffCyqNz45Pj0Ml/RGsjBl4DtzqcacMCuZT3suX39Sw64nKhcZ 1AbET0Jx4tHUbLA0AEM4JBWImI7sY4rn0xaL8ebZhgKjihq7ywS5J2RFMZGv57osn5AR q6zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849657; x=1687441657; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wlUqj841x578nzMKn5ImdhcaDhdsmsJg4wjQFvgryxE=; b=VFsonD1ju/2b9fxVNtoW65/8DXk/rU5jZ07WhXRnTq2cDmGmuhpj+FdFclQ1a72csO zWYsQrbCr5XchjQ79TGcROmJLMuVQVe5Wpeuk1u7mtvfWChzLYT/+S2TWxX7Qxuv+lN6 FrEEknVWyAsNaU6b/jSj5fRKmYb7LHSml5N3YLdPgPlHtgyjRreFLv/TDdZLXYBPQciu +S4OLaJ2pcoKJvxtEqEGgcLhBz++5xoZhPSVGK4SetW/7BezDNU1zx8ob1z5yEOoH6ne IpcV86DN8yQwss7RL20pdW73e/03Q7TzxgIQi3VqF9xQw7j+lKV/KALPxRMrNZ/h3UKR E+PQ== X-Gm-Message-State: AC+VfDzYjb/YB4KwPXJq4517icn4QrEEXdV85ngzQHW/xNKHUZ/gUuOU ndgAx0i1zb8Sv7tygGsNi/E1F44pNvEi6cEy3jk= X-Received: by 2002:a05:6a00:88b:b0:643:a6d1:b27 with SMTP id q11-20020a056a00088b00b00643a6d10b27mr23858992pfj.15.1684849656805; Tue, 23 May 2023 06:47:36 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, Juan Quintela Subject: [PATCH v2 02/27] util: Add cpuinfo-i386.c Date: Tue, 23 May 2023 06:47:08 -0700 Message-Id: <20230523134733.678646-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Add cpuinfo.h for i386 and x86_64, and the initialization for that in util/. Populate that with a slightly altered copy of the tcg host probing code. Other uses of cpuid.h will be adjusted one patch at a time. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/i386/host/cpuinfo.h | 38 ++++++++++++ host/include/x86_64/host/cpuinfo.h | 1 + util/cpuinfo-i386.c | 97 ++++++++++++++++++++++++++++++ util/meson.build | 4 ++ 4 files changed, 140 insertions(+) create mode 100644 host/include/i386/host/cpuinfo.h create mode 100644 host/include/x86_64/host/cpuinfo.h create mode 100644 util/cpuinfo-i386.c diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h new file mode 100644 index 0000000000..e6f7461378 --- /dev/null +++ b/host/include/i386/host/cpuinfo.h @@ -0,0 +1,38 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for x86. + */ + +#ifndef HOST_CPUINFO_H +#define HOST_CPUINFO_H + +/* Digested version of */ + +#define CPUINFO_ALWAYS (1u << 0) /* so cpuinfo is nonzero */ +#define CPUINFO_CMOV (1u << 1) +#define CPUINFO_MOVBE (1u << 2) +#define CPUINFO_LZCNT (1u << 3) +#define CPUINFO_POPCNT (1u << 4) +#define CPUINFO_BMI1 (1u << 5) +#define CPUINFO_BMI2 (1u << 6) +#define CPUINFO_SSE2 (1u << 7) +#define CPUINFO_SSE4 (1u << 8) +#define CPUINFO_AVX1 (1u << 9) +#define CPUINFO_AVX2 (1u << 10) +#define CPUINFO_AVX512F (1u << 11) +#define CPUINFO_AVX512VL (1u << 12) +#define CPUINFO_AVX512BW (1u << 13) +#define CPUINFO_AVX512DQ (1u << 14) +#define CPUINFO_AVX512VBMI2 (1u << 15) +#define CPUINFO_ATOMIC_VMOVDQA (1u << 16) + +/* Initialized with a constructor. */ +extern unsigned cpuinfo; + +/* + * We cannot rely on constructor ordering, so other constructors must + * use the function interface rather than the variable above. + */ +unsigned cpuinfo_init(void); + +#endif /* HOST_CPUINFO_H */ diff --git a/host/include/x86_64/host/cpuinfo.h b/host/include/x86_64/host/cpuinfo.h new file mode 100644 index 0000000000..67debab9a0 --- /dev/null +++ b/host/include/x86_64/host/cpuinfo.h @@ -0,0 +1 @@ +#include "host/include/i386/host/cpuinfo.h" diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c new file mode 100644 index 0000000000..434319aa71 --- /dev/null +++ b/util/cpuinfo-i386.c @@ -0,0 +1,97 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for x86. + */ + +#include "qemu/osdep.h" +#include "host/cpuinfo.h" +#ifdef CONFIG_CPUID_H +# include "qemu/cpuid.h" +#endif + +unsigned cpuinfo; + +/* Called both as constructor and (possibly) via other constructors. */ +unsigned __attribute__((constructor)) cpuinfo_init(void) +{ + unsigned info = cpuinfo; + + if (info) { + return info; + } + +#ifdef CONFIG_CPUID_H + unsigned max, a, b, c, d, b7 = 0, c7 = 0; + + max = __get_cpuid_max(0, 0); + + if (max >= 7) { + __cpuid_count(7, 0, a, b7, c7, d); + info |= (b7 & bit_BMI ? CPUINFO_BMI1 : 0); + info |= (b7 & bit_BMI2 ? CPUINFO_BMI2 : 0); + } + + if (max >= 1) { + __cpuid(1, a, b, c, d); + + info |= (d & bit_CMOV ? CPUINFO_CMOV : 0); + info |= (d & bit_SSE2 ? CPUINFO_SSE2 : 0); + info |= (c & bit_SSE4_1 ? CPUINFO_SSE4 : 0); + info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0); + info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0); + + /* For AVX features, we must check available and usable. */ + if ((c & bit_AVX) && (c & bit_OSXSAVE)) { + unsigned bv = xgetbv_low(0); + + if ((bv & 6) == 6) { + info |= CPUINFO_AVX1; + info |= (b7 & bit_AVX2 ? CPUINFO_AVX2 : 0); + + if ((bv & 0xe0) == 0xe0) { + info |= (b7 & bit_AVX512F ? CPUINFO_AVX512F : 0); + info |= (b7 & bit_AVX512VL ? CPUINFO_AVX512VL : 0); + info |= (b7 & bit_AVX512BW ? CPUINFO_AVX512BW : 0); + info |= (b7 & bit_AVX512DQ ? CPUINFO_AVX512DQ : 0); + info |= (c7 & bit_AVX512VBMI2 ? CPUINFO_AVX512VBMI2 : 0); + } + + /* + * The Intel SDM has added: + * Processors that enumerate support for Intel® AVX + * (by setting the feature flag CPUID.01H:ECX.AVX[bit 28]) + * guarantee that the 16-byte memory operations performed + * by the following instructions will always be carried + * out atomically: + * - MOVAPD, MOVAPS, and MOVDQA. + * - VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128. + * - VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded + * with EVEX.128 and k0 (masking disabled). + * Note that these instructions require the linear addresses + * of their memory operands to be 16-byte aligned. + * + * AMD has provided an even stronger guarantee that processors + * with AVX provide 16-byte atomicity for all cachable, + * naturally aligned single loads and stores, e.g. MOVDQU. + * + * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 + */ + __cpuid(0, a, b, c, d); + if (c == signature_INTEL_ecx || c == signature_AMD_ecx) { + info |= CPUINFO_ATOMIC_VMOVDQA; + } + } + } + } + + max = __get_cpuid_max(0x8000000, 0); + if (max >= 1) { + __cpuid(0x80000001, a, b, c, d); + info |= (c & bit_LZCNT ? CPUINFO_LZCNT : 0); + } +#endif + + info |= CPUINFO_ALWAYS; + cpuinfo = info; + return info; +} diff --git a/util/meson.build b/util/meson.build index e1f1c39e10..b3be9fad5d 100644 --- a/util/meson.build +++ b/util/meson.build @@ -108,3 +108,7 @@ if have_block endif util_ss.add(when: 'CONFIG_LINUX', if_true: files('vfio-helpers.c')) endif + +if cpu in ['x86', 'x86_64'] + util_ss.add(files('cpuinfo-i386.c')) +endif From patchwork Tue May 23 13:47:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684943 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730900wrt; Tue, 23 May 2023 06:53:19 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6eyoceY5i6AIETLFoey4ccui4zlGtfImHFQ9aQHEn+vIZX9/IILnj79XvgLR2ulx1QLdRZ X-Received: by 2002:a05:620a:2ac5:b0:75b:23a1:451 with SMTP id bn5-20020a05620a2ac500b0075b23a10451mr4290547qkb.23.1684849999010; Tue, 23 May 2023 06:53:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849998; cv=none; d=google.com; s=arc-20160816; b=bxYSGsVG109PunTRxuSnkZ60KsCeaad6oChtVheKGpz5hM9MKvEjAkQP2ZOmfM5cbm vizeFEa78hk1OTJGBhAAY/D/M++ZqNH97ttOvnz7p6ELBz0IuHyjmMPJltokhN+aTiXC k4+EV+P5LTozJ/3ai2F72CZmp5DY5S64+3bRcVl7aXP640ZVTLMyKNfqArjhOYG0redE 9KwOF4Sx5yeUx6Rk2UA9xM8irKwzfDTXDG8wODD8CZFwoKxPV+OkKRlWyeIQ5/QcbdLA 7QYRAI/uSgr8m1/wj500LE05WtwbZby0cjfi6TYGhsavIpkFNNgV2ePt8xXJG6qn/2wQ ZcmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=8wi2bOWjMyVLEFUf4QBGubplY21YXqeLF4j5hewlIPc=; b=WJ5nmWQwQthdi0ng7jY3HuhEKVgs1K09sZ5SOaSNt5uY3tQ1t5NWGhMw64RrWecBzO U+eoXv+2tR0zSq4kQISM5tOvR0VzP9Nv6rcv9zBHrFiKsFR2QeU3jaQhG5JV4WyUi+oE t1NCa++AmTMS/WyoB17vkMSTG7OZfB4GWuqnR0QHmw4+E44oYbnvdqLCf0Rt8k2gnbcc ZV04+bEjxosI/G0TpEA9CAAWWWaWDS2G8iBIu8KqVIe5/ApIMBIpM7fNUNY5ygFU8DTh kIEBYyOWv5G5MDmRd+xSgfyXT8C7B+ttpbe+uqfHQc8fzkD7zTfjIr+Beas0YC+vJaAL DeeA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fwopV+t7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ow21-20020a05620a821500b007596780d546si4844137qkn.372.2023.05.23.06.53.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:53:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fwopV+t7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMX-0008BT-Sm; Tue, 23 May 2023 09:47:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMW-0008AZ-Mu for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:40 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMV-00013T-1T for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:40 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64d18d772bdso6812695b3a.3 for ; Tue, 23 May 2023 06:47:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849658; x=1687441658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8wi2bOWjMyVLEFUf4QBGubplY21YXqeLF4j5hewlIPc=; b=fwopV+t7jzi1rJqEKM6EzBIXGv9k5IpD8MV5oyf1Ixq0li1aclNEO+UoO0+HeiQrgb jsMJubu2GePR4SoS1u4XLMMFeWMWlBNgAfBaXirNukL5QszQH86F6npKtQdwsyzKYjRH ZMDOcAMf5B52ZNgv6eeB9J4xqAegNgbbTJopeqWNjKwfsiHV8dxKIHand6NVt1RgOQn0 k1dBSVdr5WC+L++dn19HPgobZCfbfSNBCnVArpyya3d2zcNp8gYcU+Y12vmcarSIrBI0 0B/lB+388HZe0y/Gwk4QQ8SOOdTsOjDzizOsLugW53ewt1OoUar0L9NxNRnqP0EfwrOl YLxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849658; x=1687441658; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8wi2bOWjMyVLEFUf4QBGubplY21YXqeLF4j5hewlIPc=; b=IZ3jYOmSLpRJZtTQXkzW6nAl3xmxsr7zfaOGIS07brCfB5Ybd65oTpEwrLR2rGkt2r U7Kp9KsCxBm0QtxnLveJ+35YwXMBDy9dqWvB4bVcOzyq29+4U+0I/O7QBRNeT6RBy6uA jcjKJb2QzhuD9memN5JMfuhV1uibJO7+A/cRb5+wBhQdJnjAwiBNvYE66PuJB0tCWiXu 23dzDt+WYVEn1Ru9BYVAg23pBEHV0pfELLJaIJhb4g5HEDnI9VFl2U999Myl5GjkUow0 w9a31evK//fofyXzVCzlNEMFjej/sH7CmS7XAjHAQh7h/6q69A7xU16PKl1N9Zr0W5Vk /FPw== X-Gm-Message-State: AC+VfDzAfv3VdM08kDMbTTot3+fbAIII8MRuby1QbkzETtYjkh/Dpdmx 7LU12+papzAqvuO22lbfDq/T9rlATrOBkb1xrno= X-Received: by 2002:a05:6a21:33aa:b0:10a:ba3a:42dd with SMTP id yy42-20020a056a2133aa00b0010aba3a42ddmr12107134pzb.44.1684849657771; Tue, 23 May 2023 06:47:37 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 03/27] util: Add i386 CPUINFO_ATOMIC_VMOVDQU Date: Tue, 23 May 2023 06:47:09 -0700 Message-Id: <20230523134733.678646-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Add a bit to indicate when VMOVDQU is also atomic if aligned. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/i386/host/cpuinfo.h | 1 + util/cpuinfo-i386.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h index e6f7461378..a6537123cf 100644 --- a/host/include/i386/host/cpuinfo.h +++ b/host/include/i386/host/cpuinfo.h @@ -25,6 +25,7 @@ #define CPUINFO_AVX512DQ (1u << 14) #define CPUINFO_AVX512VBMI2 (1u << 15) #define CPUINFO_ATOMIC_VMOVDQA (1u << 16) +#define CPUINFO_ATOMIC_VMOVDQU (1u << 17) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c index 434319aa71..ab6143d9e7 100644 --- a/util/cpuinfo-i386.c +++ b/util/cpuinfo-i386.c @@ -77,8 +77,10 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 */ __cpuid(0, a, b, c, d); - if (c == signature_INTEL_ecx || c == signature_AMD_ecx) { + if (c == signature_INTEL_ecx) { info |= CPUINFO_ATOMIC_VMOVDQA; + } else if (c == signature_AMD_ecx) { + info |= CPUINFO_ATOMIC_VMOVDQA | CPUINFO_ATOMIC_VMOVDQU; } } } From patchwork Tue May 23 13:47:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684918 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728422wrt; Tue, 23 May 2023 06:48:01 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7aejZOL4benTBQ6fS/7UuLUfeJuiWX72TlQNBUfvpNuTOWEWSbDWvWmrZrOi5rWxNNbRuo X-Received: by 2002:a05:620a:4c84:b0:75b:23a1:8e48 with SMTP id to4-20020a05620a4c8400b0075b23a18e48mr3783758qkn.25.1684849680846; Tue, 23 May 2023 06:48:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849680; cv=none; d=google.com; s=arc-20160816; b=0tVi2cLhgDExPZCdLGBNlQtxZwjPAXdGc5P32anFGkh5zZw6F7k3Z4md7TlDY3LXa8 AxBcLVZekP8F/5egHdjb2d64ZnzfkWqRmLuLhh7xcKHeyHPFpCczu/P2XvcC0hNZNU96 GDAvGF/bU8iKA+noh4cPHGUa6YTUpq0ITKpTQpO+3AUKWUNaGrwmE+rhAXApQY4RqNCU 0ECw7wy3aC7irYDwlLo2bMvLpRZw+dXDgL/LoyTP5nFzyG3/nFAcms248Lp4J9K69OuM z69+SPdt1S5sRTCoV960VMtWZNs/w5ZXfOGN2bM5VXaktdKXVx7R+0xdwlhqy0/zUAUz AHwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=frmGIXxr84Ww41VQFYmyyhkxPgV+QZOjqj6i0A0DQ+A=; b=1LUjZd8yKidk3qVvbBV9CH1uhHNdYNx5iep/KzOPTAgOO83xg8YmoSQ9ikch4uqSxT nla3/e1liFIFozajUADvhtuGyd5K8Ss0d7vGJYcvBGnMVbP5stecCXYJ4IHSUVft5kcX zN/7JlXblB8bULVxm5KUzR25WOhmhpjTFmRe2ccy2ou71jEoov8FJCI1y3/vHqdU70dM e3b4N437zyPCC0xWWJN8P81sODWbx3sMkDUcHXbQLhuGlwIqkOu7RzoioDhtjDN/3YWV Aqr1BgwM1NxMIXwsRxtBvcaHir0WuEtKkN2EPGlHRQnlq3qZSkJbAVf5a7OCU6fUbEjd 46LA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vAycIJib; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j16-20020ae9c210000000b007590e0c550dsi4686110qkg.281.2023.05.23.06.48.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vAycIJib; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMa-0008Km-BM; Tue, 23 May 2023 09:47:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMY-0008CF-AK for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:42 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMW-00013s-6I for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:42 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-64d3bc502ddso5110115b3a.0 for ; Tue, 23 May 2023 06:47:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849659; x=1687441659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=frmGIXxr84Ww41VQFYmyyhkxPgV+QZOjqj6i0A0DQ+A=; b=vAycIJibsbqoXbokQVsguoOpH+DT0bxlHRqca2pVTSrXPx79rdzhZUgS6qvhSAe1w9 2mopA1/NBkHxAGejPGjx5i3LWMthlK2D3W62RMuIq2h3U3CyD7SMlZOy6ZOP98vr/z3s Rd5qQzpCtLj6pDmz7Y2yfbXi2O3JavJFt93tIhT9JatiOIyl0K+WnjzkQsTaxrDmGlc1 2Fe2qY7IjkVSW2m0IJCWi/8N4qoW8iyU1+gf3CC8qorWo29243lsjMQfMjNAVN9d1ltg wGP+9N8eEC8rrdlfiyYx05+5e0XPXsexTW1RinyPWwrjB438QleILIW/t/gjdNHDph7b TOug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849659; x=1687441659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=frmGIXxr84Ww41VQFYmyyhkxPgV+QZOjqj6i0A0DQ+A=; b=dLua5tUiPEz2PZV2EEHx8taWxpaxmNSuBdEf4C3wFvTSDq3e7lSZCAe4wquND01JRm BxjV0KC9aenJsTADnB3XXjMraK4TLIyDY3CTM0e5cf80iKoeFaZdhpTH/HomHXsrsEQl 0IupFCTiH2RMkrYi4Tv2cr52S9Kz9/SNvstMZ2zgPL5grIz3E8T/3duRKxalAFPfp8x1 +W0YYsjIlbuxC0K4ED6IVRaW7QfvgS99WSfGvhOyfqPq+NBxPpviEghoP289oz1a2DY8 Xx9LmUdQDB9sMSI0jwtFeEiUML720GEhq1YfqsQaS8c08fiqrK3CGH+eqirj2eoCag7i ATsA== X-Gm-Message-State: AC+VfDzVGByLnJAhBha0HhX1sgWErln+kmj+v6OzzIXfwQI5RuolTiSR KP8QEXX6r2YNn+je+ot89C7kJiPCOXuGNTDB674= X-Received: by 2002:a05:6a20:160c:b0:104:4558:b412 with SMTP id l12-20020a056a20160c00b001044558b412mr16518600pzj.25.1684849658757; Tue, 23 May 2023 06:47:38 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 04/27] tcg/i386: Use host/cpuinfo.h Date: Tue, 23 May 2023 06:47:10 -0700 Message-Id: <20230523134733.678646-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use the CPUINFO_* bits instead of the individual boolean variables that we had been using. Remove all of the init code that was moved over to cpuinfo-i386.c. Note that have_avx512* check both AVX512{F,VL}, as we had previously done during tcg_target_init. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- tcg/i386/tcg-target.h | 28 +++++---- tcg/i386/tcg-target.c.inc | 123 ++------------------------------------ 2 files changed, 22 insertions(+), 129 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 0b5a2c68c5..0106946996 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -25,6 +25,8 @@ #ifndef I386_TCG_TARGET_H #define I386_TCG_TARGET_H +#include "host/cpuinfo.h" + #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 31 @@ -111,16 +113,22 @@ typedef enum { # define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF #endif -extern bool have_bmi1; -extern bool have_popcnt; -extern bool have_avx1; -extern bool have_avx2; -extern bool have_avx512bw; -extern bool have_avx512dq; -extern bool have_avx512vbmi2; -extern bool have_avx512vl; -extern bool have_movbe; -extern bool have_atomic16; +#define have_bmi1 (cpuinfo & CPUINFO_BMI1) +#define have_popcnt (cpuinfo & CPUINFO_POPCNT) +#define have_avx1 (cpuinfo & CPUINFO_AVX1) +#define have_avx2 (cpuinfo & CPUINFO_AVX2) +#define have_movbe (cpuinfo & CPUINFO_MOVBE) +#define have_atomic16 (cpuinfo & CPUINFO_ATOMIC_VMOVDQA) + +/* + * There are interesting instructions in AVX512, so long as we have AVX512VL, + * which indicates support for EVEX on sizes smaller than 512 bits. + */ +#define have_avx512vl ((cpuinfo & CPUINFO_AVX512VL) && \ + (cpuinfo & CPUINFO_AVX512F)) +#define have_avx512bw ((cpuinfo & CPUINFO_AVX512BW) && have_avx512vl) +#define have_avx512dq ((cpuinfo & CPUINFO_AVX512DQ) && have_avx512vl) +#define have_avx512vbmi2 ((cpuinfo & CPUINFO_AVX512VBMI2) && have_avx512vl) /* optional instructions */ #define TCG_TARGET_HAS_div2_i32 1 diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 8b9a5f00e5..bfe9d98b7e 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -158,42 +158,14 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) # define SOFTMMU_RESERVE_REGS 0 #endif -/* The host compiler should supply to enable runtime features - detection, as we're not going to go so far as our own inline assembly. - If not available, default values will be assumed. */ -#if defined(CONFIG_CPUID_H) -#include "qemu/cpuid.h" -#endif - /* For 64-bit, we always know that CMOV is available. */ #if TCG_TARGET_REG_BITS == 64 -# define have_cmov 1 -#elif defined(CONFIG_CPUID_H) -static bool have_cmov; +# define have_cmov true #else -# define have_cmov 0 -#endif - -/* We need these symbols in tcg-target.h, and we can't properly conditionalize - it there. Therefore we always define the variable. */ -bool have_bmi1; -bool have_popcnt; -bool have_avx1; -bool have_avx2; -bool have_avx512bw; -bool have_avx512dq; -bool have_avx512vbmi2; -bool have_avx512vl; -bool have_movbe; -bool have_atomic16; - -#ifdef CONFIG_CPUID_H -static bool have_bmi2; -static bool have_lzcnt; -#else -# define have_bmi2 0 -# define have_lzcnt 0 +# define have_cmov (cpuinfo & CPUINFO_CMOV) #endif +#define have_bmi2 (cpuinfo & CPUINFO_BMI2) +#define have_lzcnt (cpuinfo & CPUINFO_LZCNT) static const tcg_insn_unit *tb_ret_addr; @@ -3961,93 +3933,6 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) static void tcg_target_init(TCGContext *s) { -#ifdef CONFIG_CPUID_H - unsigned a, b, c, d, b7 = 0, c7 = 0; - unsigned max = __get_cpuid_max(0, 0); - - if (max >= 7) { - /* BMI1 is available on AMD Piledriver and Intel Haswell CPUs. */ - __cpuid_count(7, 0, a, b7, c7, d); - have_bmi1 = (b7 & bit_BMI) != 0; - have_bmi2 = (b7 & bit_BMI2) != 0; - } - - if (max >= 1) { - __cpuid(1, a, b, c, d); -#ifndef have_cmov - /* For 32-bit, 99% certainty that we're running on hardware that - supports cmov, but we still need to check. In case cmov is not - available, we'll use a small forward branch. */ - have_cmov = (d & bit_CMOV) != 0; -#endif - - /* MOVBE is only available on Intel Atom and Haswell CPUs, so we - need to probe for it. */ - have_movbe = (c & bit_MOVBE) != 0; - have_popcnt = (c & bit_POPCNT) != 0; - - /* There are a number of things we must check before we can be - sure of not hitting invalid opcode. */ - if (c & bit_OSXSAVE) { - unsigned bv = xgetbv_low(0); - - if ((bv & 6) == 6) { - have_avx1 = (c & bit_AVX) != 0; - have_avx2 = (b7 & bit_AVX2) != 0; - - /* - * There are interesting instructions in AVX512, so long - * as we have AVX512VL, which indicates support for EVEX - * on sizes smaller than 512 bits. We are required to - * check that OPMASK and all extended ZMM state are enabled - * even if we're not using them -- the insns will fault. - */ - if ((bv & 0xe0) == 0xe0 - && (b7 & bit_AVX512F) - && (b7 & bit_AVX512VL)) { - have_avx512vl = true; - have_avx512bw = (b7 & bit_AVX512BW) != 0; - have_avx512dq = (b7 & bit_AVX512DQ) != 0; - have_avx512vbmi2 = (c7 & bit_AVX512VBMI2) != 0; - } - - /* - * The Intel SDM has added: - * Processors that enumerate support for Intel® AVX - * (by setting the feature flag CPUID.01H:ECX.AVX[bit 28]) - * guarantee that the 16-byte memory operations performed - * by the following instructions will always be carried - * out atomically: - * - MOVAPD, MOVAPS, and MOVDQA. - * - VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128. - * - VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded - * with EVEX.128 and k0 (masking disabled). - * Note that these instructions require the linear addresses - * of their memory operands to be 16-byte aligned. - * - * AMD has provided an even stronger guarantee that processors - * with AVX provide 16-byte atomicity for all cachable, - * naturally aligned single loads and stores, e.g. MOVDQU. - * - * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 - */ - if (have_avx1) { - __cpuid(0, a, b, c, d); - have_atomic16 = (c == signature_INTEL_ecx || - c == signature_AMD_ecx); - } - } - } - } - - max = __get_cpuid_max(0x8000000, 0); - if (max >= 1) { - __cpuid(0x80000001, a, b, c, d); - /* LZCNT was introduced with AMD Barcelona and Intel Haswell CPUs. */ - have_lzcnt = (c & bit_LZCNT) != 0; - } -#endif /* CONFIG_CPUID_H */ - tcg_target_available_regs[TCG_TYPE_I32] = ALL_GENERAL_REGS; if (TCG_TARGET_REG_BITS == 64) { tcg_target_available_regs[TCG_TYPE_I64] = ALL_GENERAL_REGS; From patchwork Tue May 23 13:47:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684916 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728354wrt; Tue, 23 May 2023 06:47:49 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5j5Y4vMG3qLWQc+Mm+LKdBxsNK/69Yu1+SXJXA3MzSlljnLr2IDLCwMr7G8s2O8Y95d/tR X-Received: by 2002:a37:405:0:b0:75b:23a1:d857 with SMTP id 5-20020a370405000000b0075b23a1d857mr4767943qke.25.1684849669506; Tue, 23 May 2023 06:47:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849669; cv=none; d=google.com; s=arc-20160816; b=t9BarJ4ttcaNveIXKDNIlI1AHv5msP8GxRMaEOyhidrXomQripE80FWui8wJ+84+rO DTZp6dBXEZOaekM8JAot7N10cLVxQjaKuP+j1u5uhFEFFt6S1H+nKt1okr6JZnAIcZ1f PIfc/ozDZ99hNFPs3zo6GWbPpLPR2btzXfaH1csdBm/TlTKOjN5XWYEwC8kbpJI/7t65 0CPwH//2JR3yNvQa4yw6D5rL4La+Kv2xP2lDyFhDFFGm3YyzIaixgS+BfwSYP0b3Gw7g +uClGESNGeWcWbxc9njGFBgLRp/995R/g23ukLKDGs4V6cBKl/9aLs+4AYMWdqvQR8l6 uBbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=WcxZjIY/T80MjuA6OZ00h+5fCLAwNHjF9M1KXt4rlvU=; b=DWZBXiizVBdxN++8eieQbsSYO6WisrIt/oxXlQialV4t68aAPsWywxGxoS9odaBSPX IhMTXm9ddhiyLprxv3bmUol9vOaBoY9X/VQ/zZZP1qSIyg0Esc2BX2KIifdnsFbxJQ/z cFiw/GVoF/vHNgz+qg9MtOcgfVBH4hshAPEw+4cimcaTNhnKTxoy/EsRjeNlnugXnBMl 9nOYYuBKkM9XYngJKB3/xGAbxO9UphtZCpEfImp6z+Nj/F/kWVIf/gMi7tH2tWF1vRta budIVHi+qn53nm0z9FOMh+SqvCUgODVQOYW0rI+IFKu7ezGaBpf5ge3P6zT5wQC5F8Ns PU1A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Ups/My3t"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z15-20020ac87f8f000000b003e6415666b6si5094924qtj.163.2023.05.23.06.47.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:47:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Ups/My3t"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMb-0008Qf-75; Tue, 23 May 2023 09:47:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMY-0008Cc-P2 for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:42 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMW-000141-V8 for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:42 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-64d44b198baso2519331b3a.0 for ; Tue, 23 May 2023 06:47:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849659; x=1687441659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WcxZjIY/T80MjuA6OZ00h+5fCLAwNHjF9M1KXt4rlvU=; b=Ups/My3t8avsQ85Rg7lt4VqQTExlV/NL4MSSXtzQb3il+0uLuKnzYlCu+9r3UucWzn FGYwZWRCTiuXf0I3hGrYf/TUE4HN8I+RJlsd40WbTZuBb8XmEJQX3o7VH4JVaYarCKcf 8/Vo5DhN1niyXOfimJYP0seVOJPHG2O2blR4SczjbU9cJ/K6SOHXIOCkr8OEAe/4eF8M ZgRLbo2YEYgGmewjYZ867jdldRo3zYCSud6pvgTrRluqKpbsF32Orx2MqPFaJaxZpigr fVhR9G4tBsLpjBbTUWfExQZyKSp+aWpv/OjJfNrXOCJ8UpDeqM/LNnfLuux8llzIHJtP k9Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849659; x=1687441659; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WcxZjIY/T80MjuA6OZ00h+5fCLAwNHjF9M1KXt4rlvU=; b=UszpwDSlj4eaLqsMomjv/O+o7k4TkB9Xmc//uVI3pScdxAFwA40ej6LcUnCRiwBtCj Qxxmtv2ATULg2yz6JPH+36yho7vT2pq/kId8kiiE2pBpIRtQQsmlLvnfK7Kj5y81ZBqB I/nxGEgmplMCvC8ztc+CroLKwxglXlYDLjz6FKYNvnIpMvNtRf9gCu4Kz9kgFLZMmsmd ePyzRGVfEv4hiSvvcnvkEV8xWx715r2gf96EcbyQwcdbaM7qGzSe5fCanTU6BtpmR6ap BqyOFkb1geaqGz/P5qK2S0GsSGerVzp3kdJZNtSMcIgdMHaYyy+YgT2xEPhXrSfYRkrU t3Sg== X-Gm-Message-State: AC+VfDzMxJtOlGaTHJGchP97Cy21CaWBseXDtjizuaiBpuimT/og9ftI Ynk6QNEWp9kzXZBObtp/kMVwhFv9GhweZtfblXQ= X-Received: by 2002:a05:6a00:1250:b0:63b:19e5:a96e with SMTP id u16-20020a056a00125000b0063b19e5a96emr13520990pfi.9.1684849659524; Tue, 23 May 2023 06:47:39 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 05/27] util/bufferiszero: Use i386 host/cpuinfo.h Date: Tue, 23 May 2023 06:47:11 -0700 Message-Id: <20230523134733.678646-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use cpuinfo_init() during init_accel(), and the variable cpuinfo during test_buffer_is_zero_next_accel(). Adjust the logic that cycles through the set of accelerators for testing. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- util/bufferiszero.c | 126 ++++++++++++++++---------------------------- 1 file changed, 45 insertions(+), 81 deletions(-) diff --git a/util/bufferiszero.c b/util/bufferiszero.c index 1886bc5ba4..d3c14320ef 100644 --- a/util/bufferiszero.c +++ b/util/bufferiszero.c @@ -24,6 +24,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/bswap.h" +#include "host/cpuinfo.h" static bool buffer_zero_int(const void *buf, size_t len) @@ -184,111 +185,74 @@ buffer_zero_avx512(const void *buf, size_t len) } #endif /* CONFIG_AVX512F_OPT */ - -/* Note that for test_buffer_is_zero_next_accel, the most preferred - * ISA must have the least significant bit. - */ -#define CACHE_AVX512F 1 -#define CACHE_AVX2 2 -#define CACHE_SSE4 4 -#define CACHE_SSE2 8 - -/* Make sure that these variables are appropriately initialized when +/* + * Make sure that these variables are appropriately initialized when * SSE2 is enabled on the compiler command-line, but the compiler is * too old to support CONFIG_AVX2_OPT. */ #if defined(CONFIG_AVX512F_OPT) || defined(CONFIG_AVX2_OPT) -# define INIT_CACHE 0 -# define INIT_ACCEL buffer_zero_int +# define INIT_USED 0 +# define INIT_LENGTH 0 +# define INIT_ACCEL buffer_zero_int #else # ifndef __SSE2__ # error "ISA selection confusion" # endif -# define INIT_CACHE CACHE_SSE2 -# define INIT_ACCEL buffer_zero_sse2 +# define INIT_USED CPUINFO_SSE2 +# define INIT_LENGTH 64 +# define INIT_ACCEL buffer_zero_sse2 #endif -static unsigned cpuid_cache = INIT_CACHE; +static unsigned used_accel = INIT_USED; +static unsigned length_to_accel = INIT_LENGTH; static bool (*buffer_accel)(const void *, size_t) = INIT_ACCEL; -static int length_to_accel = 64; -static void init_accel(unsigned cache) +static unsigned __attribute__((noinline)) +select_accel_cpuinfo(unsigned info) { - bool (*fn)(const void *, size_t) = buffer_zero_int; - if (cache & CACHE_SSE2) { - fn = buffer_zero_sse2; - length_to_accel = 64; - } -#ifdef CONFIG_AVX2_OPT - if (cache & CACHE_SSE4) { - fn = buffer_zero_sse4; - length_to_accel = 64; - } - if (cache & CACHE_AVX2) { - fn = buffer_zero_avx2; - length_to_accel = 128; - } -#endif + static const struct { + unsigned bit; + unsigned len; + bool (*fn)(const void *, size_t); + } all[] = { #ifdef CONFIG_AVX512F_OPT - if (cache & CACHE_AVX512F) { - fn = buffer_zero_avx512; - length_to_accel = 256; - } + { CPUINFO_AVX512F, 256, buffer_zero_avx512 }, #endif - buffer_accel = fn; +#ifdef CONFIG_AVX2_OPT + { CPUINFO_AVX2, 128, buffer_zero_avx2 }, + { CPUINFO_SSE4, 64, buffer_zero_sse4 }, +#endif + { CPUINFO_SSE2, 64, buffer_zero_sse2 }, + { CPUINFO_ALWAYS, 0, buffer_zero_int }, + }; + + for (unsigned i = 0; i < ARRAY_SIZE(all); ++i) { + if (info & all[i].bit) { + length_to_accel = all[i].len; + buffer_accel = all[i].fn; + return all[i].bit; + } + } + return 0; } #if defined(CONFIG_AVX512F_OPT) || defined(CONFIG_AVX2_OPT) -#include "qemu/cpuid.h" - -static void __attribute__((constructor)) init_cpuid_cache(void) +static void __attribute__((constructor)) init_accel(void) { - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - unsigned cache = 0; - - if (max >= 1) { - __cpuid(1, a, b, c, d); - if (d & bit_SSE2) { - cache |= CACHE_SSE2; - } - if (c & bit_SSE4_1) { - cache |= CACHE_SSE4; - } - - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - unsigned bv = xgetbv_low(0); - __cpuid_count(7, 0, a, b, c, d); - if ((bv & 0x6) == 0x6 && (b & bit_AVX2)) { - cache |= CACHE_AVX2; - } - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512F)) { - cache |= CACHE_AVX512F; - } - } - } - cpuid_cache = cache; - init_accel(cache); + used_accel = select_accel_cpuinfo(cpuinfo_init()); } #endif /* CONFIG_AVX2_OPT */ bool test_buffer_is_zero_next_accel(void) { - /* If no bits set, we just tested buffer_zero_int, and there - are no more acceleration options to test. */ - if (cpuid_cache == 0) { - return false; - } - /* Disable the accelerator we used before and select a new one. */ - cpuid_cache &= cpuid_cache - 1; - init_accel(cpuid_cache); - return true; + /* + * Accumulate the accelerators that we've already tested, and + * remove them from the set to test this round. We'll get back + * a zero from select_accel_cpuinfo when there are no more. + */ + unsigned used = select_accel_cpuinfo(cpuinfo & ~used_accel); + used_accel |= used; + return used; } static bool select_accel_fn(const void *buf, size_t len) From patchwork Tue May 23 13:47:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684922 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728652wrt; Tue, 23 May 2023 06:48:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6n6ETZOvYRkoZg+ndHvCOJ6t8gmEyYFPY39ZNer/UxiCFNJObzWrXZaYEIR2PJYFomA2mI X-Received: by 2002:ac8:5751:0:b0:3f6:aff0:6dec with SMTP id 17-20020ac85751000000b003f6aff06decmr10538390qtx.32.1684849707441; Tue, 23 May 2023 06:48:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849707; cv=none; d=google.com; s=arc-20160816; b=O1qFINWaUTw3ym0hOw2aUeUkPsPDnQlTTj6+RSrrJUwdcztzWVODcQaGiuaTwmmY6N mK3B+WN2hPHv2+jqnWBmzzQ88647S2wuTZG3+si7O0cIzS9GzvjKwzE0fcGshPQtn2Xe ndkIbL32DU//JM4VOYRp/xljcOu0nsYleXWRu17Wv3gE6wIGb1XwX6I8icFgYNqIp87R n87kPaqOeNo9SHmoZ8wSIB1gmV+CccCXGdUAjNZTCgIvwXmFmY6GYZOOmu5V4vDstWjE gIzDQMDMFvB3H8koh/MWH7bV5KN5zcr7arwknzt34Vb2AOgPI/mn/0NUgDSeR46WBgGR mrdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=TCEmCa6svz6iI89FPgua0fOjkJEldQJYp9iT0pK2PQY=; b=FTl1OE3D4CZ4vzVmj3UBVKNFBv3wy3HP6OnM1QGMzw5xEouJ2xbgunFGFXZXRTmcKW Yq2ZJ3qqnECjZqDpVNysjSKWb9DM14yW0BU4K2bPUI3DlUijTjYBADqaRnCpS2VVqFIg mF31OpDyGuadQZjOmKm52mjqcrr1xn39Du5Mo2bqIug7d/Wj27OzU4hQvbbFoCEjO7dx aL4ytipJYiRMru6qXy7fJwmB+pEo8Lou1Wddpvy6DDBgLJ8MDfevFgTXRoxuJjI3oH/f Jz5BrtbwNoW43a+mAWBsbJyKjH0l1bh7b5fNrNqdUtCkwHdopxqMSjOC7NrdZBtth1HP YsFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iRRqx5Bk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id gy9-20020a056214242900b00616000fbfbfsi5085065qvb.321.2023.05.23.06.48.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=iRRqx5Bk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMc-0008Ud-Ck; Tue, 23 May 2023 09:47:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMZ-0008Ev-BN for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:43 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMX-00012o-B3 for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:43 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64d3bc502ddso5110144b3a.0 for ; Tue, 23 May 2023 06:47:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849660; x=1687441660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TCEmCa6svz6iI89FPgua0fOjkJEldQJYp9iT0pK2PQY=; b=iRRqx5BkHwsP+gTnyIGLFc3+0Uw4hw/B26V6zPewwX2tt6udOzwGM5V/mZDcaxoP8f 1TALmSUCpfPSWjMgzf4zp474sOO+ERfaTlEEw2QlXRYcddYi9PrAZySn9rsRLRvqnBNb r+RVWcfelBg7IrR98w0pTrigd7vqWLIGWEeTJdUXiIIDBeSne1pVnpG3QJfL5Qsu9xGR rF3V6wqCNt01qSKS7HAyT+AbaBW9dhycgwI98t1V3nbKNPQjI+349IyqQEcKQR634Uxj zIEVit2gxMBa1hP+iI3TNk7RPXVXK0Ry/MJhrA2jnLAnn5yd4yzvKsMTq4Hsd1ml5Ggz WxAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849660; x=1687441660; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TCEmCa6svz6iI89FPgua0fOjkJEldQJYp9iT0pK2PQY=; b=d878vU4JcU/i6J9PuohnEEzX9RAR7xmYo7m3UJETzO265vtudytlPM4Nth56Lt0VR/ 6JRWJToC2mRUsdKq6BuTksodmEdG0MmFLl24eUcsXiRULUB1bNM7Hf4U45L4oWEwwQz3 UU7G4uVYJzBjChlPYCtvTaaRpDuIoMf2rQSpdhqoiqjde3G3+ElaNn/BD50kQFqc6TzZ 1JYRQpeX8zoV5Rp49q7OAe3NkdQKo/yY0kImGUO7P+CCTgJF14ebPy87e5U9gCSUfE8I Ag4AF4Nav2nc1sZls8SkBstxo2YcPzOPJpSynln+h+V+BvIymGD4JUcxp649Zy9tGZJt 3jQA== X-Gm-Message-State: AC+VfDxGBpiOVpOD3Mz90QqGhB+MTlBdJ2J/Zdo2m9nLL3dHAfm4qdHj VqluF+qwBVbvxZVg6ussaQmc0PSt4bCGT2ep8P8= X-Received: by 2002:a05:6a00:1741:b0:64d:42f6:4c7b with SMTP id j1-20020a056a00174100b0064d42f64c7bmr15494604pfc.27.1684849660554; Tue, 23 May 2023 06:47:40 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, Juan Quintela Subject: [PATCH v2 06/27] migration/xbzrle: Shuffle function order Date: Tue, 23 May 2023 06:47:12 -0700 Message-Id: <20230523134733.678646-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Place the CONFIG_AVX512BW_OPT block at the top, which will aid function selection in the next patch. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- migration/xbzrle.c | 244 ++++++++++++++++++++++----------------------- 1 file changed, 122 insertions(+), 122 deletions(-) diff --git a/migration/xbzrle.c b/migration/xbzrle.c index 258e4959c9..751b5428f7 100644 --- a/migration/xbzrle.c +++ b/migration/xbzrle.c @@ -15,6 +15,128 @@ #include "qemu/host-utils.h" #include "xbzrle.h" +#if defined(CONFIG_AVX512BW_OPT) +#include + +int __attribute__((target("avx512bw"))) +xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, + uint8_t *dst, int dlen) +{ + uint32_t zrun_len = 0, nzrun_len = 0; + int d = 0, i = 0, num = 0; + uint8_t *nzrun_start = NULL; + /* add 1 to include residual part in main loop */ + uint32_t count512s = (slen >> 6) + 1; + /* countResidual is tail of data, i.e., countResidual = slen % 64 */ + uint32_t count_residual = slen & 0b111111; + bool never_same = true; + uint64_t mask_residual = 1; + mask_residual <<= count_residual; + mask_residual -= 1; + __m512i r = _mm512_set1_epi32(0); + + while (count512s) { + int bytes_to_check = 64; + uint64_t mask = 0xffffffffffffffff; + if (count512s == 1) { + bytes_to_check = count_residual; + mask = mask_residual; + } + __m512i old_data = _mm512_mask_loadu_epi8(r, + mask, old_buf + i); + __m512i new_data = _mm512_mask_loadu_epi8(r, + mask, new_buf + i); + uint64_t comp = _mm512_cmpeq_epi8_mask(old_data, new_data); + count512s--; + + bool is_same = (comp & 0x1); + while (bytes_to_check) { + if (d + 2 > dlen) { + return -1; + } + if (is_same) { + if (nzrun_len) { + d += uleb128_encode_small(dst + d, nzrun_len); + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start = new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d += nzrun_len; + nzrun_len = 0; + } + /* 64 data at a time for speed */ + if (count512s && (comp == 0xffffffffffffffff)) { + i += 64; + zrun_len += 64; + break; + } + never_same = false; + num = ctz64(~comp); + num = (num < bytes_to_check) ? num : bytes_to_check; + zrun_len += num; + bytes_to_check -= num; + comp >>= num; + i += num; + if (bytes_to_check) { + /* still has different data after same data */ + d += uleb128_encode_small(dst + d, zrun_len); + zrun_len = 0; + } else { + break; + } + } + if (never_same || zrun_len) { + /* + * never_same only acts if + * data begins with diff in first count512s + */ + d += uleb128_encode_small(dst + d, zrun_len); + zrun_len = 0; + never_same = false; + } + /* has diff, 64 data at a time for speed */ + if ((bytes_to_check == 64) && (comp == 0x0)) { + i += 64; + nzrun_len += 64; + break; + } + num = ctz64(comp); + num = (num < bytes_to_check) ? num : bytes_to_check; + nzrun_len += num; + bytes_to_check -= num; + comp >>= num; + i += num; + if (bytes_to_check) { + /* mask like 111000 */ + d += uleb128_encode_small(dst + d, nzrun_len); + /* overflow */ + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start = new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d += nzrun_len; + nzrun_len = 0; + is_same = true; + } + } + } + + if (nzrun_len != 0) { + d += uleb128_encode_small(dst + d, nzrun_len); + /* overflow */ + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start = new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d += nzrun_len; + } + return d; +} +#endif + /* page = zrun nzrun | zrun nzrun page @@ -175,125 +297,3 @@ int xbzrle_decode_buffer(uint8_t *src, int slen, uint8_t *dst, int dlen) return d; } - -#if defined(CONFIG_AVX512BW_OPT) -#include - -int __attribute__((target("avx512bw"))) -xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, - uint8_t *dst, int dlen) -{ - uint32_t zrun_len = 0, nzrun_len = 0; - int d = 0, i = 0, num = 0; - uint8_t *nzrun_start = NULL; - /* add 1 to include residual part in main loop */ - uint32_t count512s = (slen >> 6) + 1; - /* countResidual is tail of data, i.e., countResidual = slen % 64 */ - uint32_t count_residual = slen & 0b111111; - bool never_same = true; - uint64_t mask_residual = 1; - mask_residual <<= count_residual; - mask_residual -= 1; - __m512i r = _mm512_set1_epi32(0); - - while (count512s) { - int bytes_to_check = 64; - uint64_t mask = 0xffffffffffffffff; - if (count512s == 1) { - bytes_to_check = count_residual; - mask = mask_residual; - } - __m512i old_data = _mm512_mask_loadu_epi8(r, - mask, old_buf + i); - __m512i new_data = _mm512_mask_loadu_epi8(r, - mask, new_buf + i); - uint64_t comp = _mm512_cmpeq_epi8_mask(old_data, new_data); - count512s--; - - bool is_same = (comp & 0x1); - while (bytes_to_check) { - if (d + 2 > dlen) { - return -1; - } - if (is_same) { - if (nzrun_len) { - d += uleb128_encode_small(dst + d, nzrun_len); - if (d + nzrun_len > dlen) { - return -1; - } - nzrun_start = new_buf + i - nzrun_len; - memcpy(dst + d, nzrun_start, nzrun_len); - d += nzrun_len; - nzrun_len = 0; - } - /* 64 data at a time for speed */ - if (count512s && (comp == 0xffffffffffffffff)) { - i += 64; - zrun_len += 64; - break; - } - never_same = false; - num = ctz64(~comp); - num = (num < bytes_to_check) ? num : bytes_to_check; - zrun_len += num; - bytes_to_check -= num; - comp >>= num; - i += num; - if (bytes_to_check) { - /* still has different data after same data */ - d += uleb128_encode_small(dst + d, zrun_len); - zrun_len = 0; - } else { - break; - } - } - if (never_same || zrun_len) { - /* - * never_same only acts if - * data begins with diff in first count512s - */ - d += uleb128_encode_small(dst + d, zrun_len); - zrun_len = 0; - never_same = false; - } - /* has diff, 64 data at a time for speed */ - if ((bytes_to_check == 64) && (comp == 0x0)) { - i += 64; - nzrun_len += 64; - break; - } - num = ctz64(comp); - num = (num < bytes_to_check) ? num : bytes_to_check; - nzrun_len += num; - bytes_to_check -= num; - comp >>= num; - i += num; - if (bytes_to_check) { - /* mask like 111000 */ - d += uleb128_encode_small(dst + d, nzrun_len); - /* overflow */ - if (d + nzrun_len > dlen) { - return -1; - } - nzrun_start = new_buf + i - nzrun_len; - memcpy(dst + d, nzrun_start, nzrun_len); - d += nzrun_len; - nzrun_len = 0; - is_same = true; - } - } - } - - if (nzrun_len != 0) { - d += uleb128_encode_small(dst + d, nzrun_len); - /* overflow */ - if (d + nzrun_len > dlen) { - return -1; - } - nzrun_start = new_buf + i - nzrun_len; - memcpy(dst + d, nzrun_start, nzrun_len); - d += nzrun_len; - } - return d; -} -#endif From patchwork Tue May 23 13:47:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684931 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729797wrt; Tue, 23 May 2023 06:50:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5VpOt05StQbMShxikRimttNsoE1Av4ZtwrzxzdxDE2v+rx8vYvmbSGDOqZX2ZxkFtktwlf X-Received: by 2002:a37:308:0:b0:75b:23a1:403 with SMTP id 8-20020a370308000000b0075b23a10403mr3889568qkd.25.1684849854914; Tue, 23 May 2023 06:50:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849854; cv=none; d=google.com; s=arc-20160816; b=Ug2egK9idmduPeqw7PqQzhGhrocvyvRZMuI+xRuXoUslvUAtiRjfPj31G/yZLWnhDu e2bjbQnOp0vqSDUGaK5quKFWumGG5Ztn4PluZEJ6ACkJYy+OvzPqMi+Ws4uhNrgoGcVN 7b3bUW8DlkeWLwhHg6LBY0nUN3fFRHnFkj2O4axjpS7Wk2pc0cbxIpJLV4X+CfmhvB9T MVlsQR7jsjQfRuTJB7u8itgncLJAfJ3HISGxtvJmsHydcn0b87Xkkxtfs/cW1jZ7q7UH vlgQE9Ec7FOdoUH4lO3neXeF5SLgKtKkdJq9AficD1hSAhC2vsiQLODJfjeZywwvZCFd RXDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ihz9eMnKafBCXMKR5tA0Mrs4r+NsNN8WEkICVzjP6LI=; b=IigvMZrLxMOgJsu8ME3GfdtGUTqVUndspLigM/Lx9YouA9afCVuqK95YSokN+3gloD L2DVOj2zv+RhrjalpUUADBDxzCAPEksOs4ScIJVcBm4BRA94qwcTAOtdgHuzWtdg09Vk FTICF+F2bSixNkxS2P+07/ditzblctkgjsZIljUzqCup2oTPpR5sX6zdJigtErKIpW8o E/Lj5cz+NQ3VHTwUigFc8ZD9VasboI2l4TurODkUkgkX2/rC3aYJACfJ0OAbeN1g/ZwB xBYoTY6qRVbjQ/7jkyTo0VfnxzF1adacRrYY0/qNkA+uiXCfhysRFHG0+cdrYy+wPt3Q PvRw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=b1KSTqFm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k13-20020ae9f10d000000b0075783b0d4f6si4798562qkg.442.2023.05.23.06.50.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:50:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=b1KSTqFm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMf-0000Hc-O1; Tue, 23 May 2023 09:47:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMd-00005m-Um for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:47 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMY-00014T-UC for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:46 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-64d4e4598f0so3729036b3a.2 for ; Tue, 23 May 2023 06:47:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849661; x=1687441661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ihz9eMnKafBCXMKR5tA0Mrs4r+NsNN8WEkICVzjP6LI=; b=b1KSTqFmZMnVBWtwebGwzz6wurvniUqfFtaeX+FT+FR+UwVJ63E8TgnmrCI2ZWr+nt vditVh9ruZldbaGrT1TqOSTv1c3CDN3x/4O5NXhoPcPCY5Q5oOWBGpb0q1DY37teE6Ze WYjzrRfgakdzE1xeZkPzPBDlCCYyX1mXX1DVNnU1liJ8HnvRMBSCBo3ULs1vUbYzNNI7 VmAhky2qewpxAm1B7z4IY8uB7HZlsUcgyYYNBvq6JiEKg822aSrlQsN2cZ2QMboDlSM6 Yb7ffH6fwujD/Go5lzd4gJHbv8/1b7FFLN7jy0mILealK02ifEUttlK6S4G7APjpnYNN djGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849661; x=1687441661; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ihz9eMnKafBCXMKR5tA0Mrs4r+NsNN8WEkICVzjP6LI=; b=QolRojassKWHjkPBRZBRLg+RayJX3z6gpmxCBxLfc8e3zR3nDHMo9XshBTypfhSrVm t9BltPc0GV8oYRqqWPmUWlt358/a7MwOCMWhvHe7ta/IMJf2oFjuAhYCM7MImHZ5Jlgm P9fMRxJAXz1oyAkQoQtrQcbMMjLUcrVIEdEuo1EHxPxUDQ4ilremQjVZ0jFP723SSABP DamVoj2aRMNHNLyL+S9i6XdBUf8t+YXO0R0iJnW4uJBK10/TOZ3INFwJGblabiuUQnGV YEDzfU8QnTl7y54owNakNTN1VYI5tS19bLogD4swWf/ivbovr77dtcl8EXFeWFkBXyJG BuoA== X-Gm-Message-State: AC+VfDzsPF+bYXoGptk/HmEvsACVlMzPwe0NeTCHXifwDScccEsX6gor a6R1jzzVq0SQieN+hvCr/0fsmiUdna29em4Fx8A= X-Received: by 2002:a05:6a00:1949:b0:64c:c5f9:152a with SMTP id s9-20020a056a00194900b0064cc5f9152amr17906337pfk.23.1684849661369; Tue, 23 May 2023 06:47:41 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, Juan Quintela Subject: [PATCH v2 07/27] migration/xbzrle: Use i386 host/cpuinfo.h Date: Tue, 23 May 2023 06:47:13 -0700 Message-Id: <20230523134733.678646-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Perform the function selection once, and only if CONFIG_AVX512_OPT is enabled. Centralize the selection to xbzrle.c, instead of spreading the init across 3 files. Remove xbzrle-bench.c. The benefit of being able to benchmark the different implementations is less important than not peeking into the internals of the implementation. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- migration/xbzrle.h | 5 +- migration/ram.c | 34 +-- migration/xbzrle.c | 26 +- tests/bench/xbzrle-bench.c | 469 ------------------------------------- tests/unit/test-xbzrle.c | 49 +--- tests/bench/meson.build | 6 - 6 files changed, 39 insertions(+), 550 deletions(-) delete mode 100644 tests/bench/xbzrle-bench.c diff --git a/migration/xbzrle.h b/migration/xbzrle.h index 6feb49160a..39e651b9ec 100644 --- a/migration/xbzrle.h +++ b/migration/xbzrle.h @@ -18,8 +18,5 @@ int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen, uint8_t *dst, int dlen); int xbzrle_decode_buffer(uint8_t *src, int slen, uint8_t *dst, int dlen); -#if defined(CONFIG_AVX512BW_OPT) -int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, - uint8_t *dst, int dlen); -#endif + #endif diff --git a/migration/ram.c b/migration/ram.c index 9fb076fa58..88a6c82e63 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -90,34 +90,6 @@ #define RAM_SAVE_FLAG_MULTIFD_FLUSH 0x200 /* We can't use any flag that is bigger than 0x200 */ -int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, - uint8_t *, int) = xbzrle_encode_buffer; -#if defined(CONFIG_AVX512BW_OPT) -#include "qemu/cpuid.h" -static void __attribute__((constructor)) init_cpu_flag(void) -{ - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - if (max >= 1) { - __cpuid(1, a, b, c, d); - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - int bv; - __asm("xgetbv" : "=a"(bv), "=d"(d) : "c"(0)); - __cpuid_count(7, 0, a, b, c, d); - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512BW)) { - xbzrle_encode_buffer_func = xbzrle_encode_buffer_avx512; - } - } - } -} -#endif - XBZRLECacheStats xbzrle_counters; /* used by the search for pages to send */ @@ -660,9 +632,9 @@ static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss, memcpy(XBZRLE.current_buf, *current_data, TARGET_PAGE_SIZE); /* XBZRLE encoding (if there is no overflow) */ - encoded_len = xbzrle_encode_buffer_func(prev_cached_page, XBZRLE.current_buf, - TARGET_PAGE_SIZE, XBZRLE.encoded_buf, - TARGET_PAGE_SIZE); + encoded_len = xbzrle_encode_buffer(prev_cached_page, XBZRLE.current_buf, + TARGET_PAGE_SIZE, XBZRLE.encoded_buf, + TARGET_PAGE_SIZE); /* * Update the cache contents, so that it corresponds to the data diff --git a/migration/xbzrle.c b/migration/xbzrle.c index 751b5428f7..3eddcf249b 100644 --- a/migration/xbzrle.c +++ b/migration/xbzrle.c @@ -17,8 +17,9 @@ #if defined(CONFIG_AVX512BW_OPT) #include +#include "host/cpuinfo.h" -int __attribute__((target("avx512bw"))) +static int __attribute__((target("avx512bw"))) xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, uint8_t *dst, int dlen) { @@ -135,6 +136,29 @@ xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, } return d; } + +static int xbzrle_encode_buffer_int(uint8_t *old_buf, uint8_t *new_buf, + int slen, uint8_t *dst, int dlen); + +static int (*accel_func)(uint8_t *, uint8_t *, int, uint8_t *, int); + +static void __attribute__((constructor)) init_accel(void) +{ + unsigned info = cpuinfo_init(); + if (info & CPUINFO_AVX512BW) { + accel_func = xbzrle_encode_buffer_avx512; + } else { + accel_func = xbzrle_encode_buffer_int; + } +} + +int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen, + uint8_t *dst, int dlen) +{ + return accel_func(old_buf, new_buf, slen, dst, dlen); +} + +#define xbzrle_encode_buffer xbzrle_encode_buffer_int #endif /* diff --git a/tests/bench/xbzrle-bench.c b/tests/bench/xbzrle-bench.c deleted file mode 100644 index 8848a3a32d..0000000000 --- a/tests/bench/xbzrle-bench.c +++ /dev/null @@ -1,469 +0,0 @@ -/* - * Xor Based Zero Run Length Encoding unit tests. - * - * Copyright 2013 Red Hat, Inc. and/or its affiliates - * - * Authors: - * Orit Wasserman - * - * This work is licensed under the terms of the GNU GPL, version 2 or later. - * See the COPYING file in the top-level directory. - * - */ -#include "qemu/osdep.h" -#include "qemu/cutils.h" -#include "../migration/xbzrle.h" - -#if defined(CONFIG_AVX512BW_OPT) -#define XBZRLE_PAGE_SIZE 4096 -static bool is_cpu_support_avx512bw; -#include "qemu/cpuid.h" -static void __attribute__((constructor)) init_cpu_flag(void) -{ - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - is_cpu_support_avx512bw = false; - if (max >= 1) { - __cpuid(1, a, b, c, d); - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - int bv; - __asm("xgetbv" : "=a"(bv), "=d"(d) : "c"(0)); - __cpuid_count(7, 0, a, b, c, d); - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512BW)) { - is_cpu_support_avx512bw = true; - } - } - } - return ; -} - -struct ResTime { - float t_raw; - float t_512; -}; - - -/* Function prototypes -int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, - uint8_t *dst, int dlen); -*/ -static void encode_decode_zero(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0; - int dlen = 0, dlen512 = 0; - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); - - for (i = diff_len; i > 0; i--) { - buffer[1000 + i] = i; - buffer512[1000 + i] = i; - } - - buffer[1000 + diff_len + 3] = 103; - buffer[1000 + diff_len + 5] = 105; - - buffer512[1000 + diff_len + 3] = 103; - buffer512[1000 + diff_len + 5] = 105; - - /* encode zero page */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(buffer, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(dlen == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(buffer512, buffer512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(dlen512 == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(buffer512); - g_free(compressed512); - -} - -static void test_encode_decode_zero_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_zero(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Zero test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_unchanged(struct ResTime *res) -{ - uint8_t *compressed = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0; - int dlen = 0, dlen512 = 0; - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); - - for (i = diff_len; i > 0; i--) { - test[1000 + i] = i + 4; - test512[1000 + i] = i + 4; - } - - test[1000 + diff_len + 3] = 107; - test[1000 + diff_len + 5] = 109; - - test512[1000 + diff_len + 3] = 107; - test512[1000 + diff_len + 5] = 109; - - /* test unchanged buffer */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(test, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(dlen == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(test512, test512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(dlen512 == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(test); - g_free(compressed); - g_free(test512); - g_free(compressed512); - -} - -static void test_encode_decode_unchanged_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_unchanged(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Unchanged test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_1_byte(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc(XBZRLE_PAGE_SIZE); - int dlen = 0, rc = 0, dlen512 = 0, rc512 = 0; - uint8_t buf[2]; - uint8_t buf512[2]; - - test[XBZRLE_PAGE_SIZE - 1] = 1; - test512[XBZRLE_PAGE_SIZE - 1] = 1; - - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(dlen == (uleb128_encode_small(&buf[0], 4095) + 2)); - - rc = xbzrle_decode_buffer(compressed, dlen, buffer, XBZRLE_PAGE_SIZE); - g_assert(rc == XBZRLE_PAGE_SIZE); - g_assert(memcmp(test, buffer, XBZRLE_PAGE_SIZE) == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(dlen512 == (uleb128_encode_small(&buf512[0], 4095) + 2)); - - rc512 = xbzrle_decode_buffer(compressed512, dlen512, buffer512, - XBZRLE_PAGE_SIZE); - g_assert(rc512 == XBZRLE_PAGE_SIZE); - g_assert(memcmp(test512, buffer512, XBZRLE_PAGE_SIZE) == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_1_byte_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_1_byte(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("1 byte test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_overflow(struct ResTime *res) -{ - uint8_t *compressed = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0, rc = 0, rc512 = 0; - - for (i = 0; i < XBZRLE_PAGE_SIZE / 2 - 1; i++) { - test[i * 2] = 1; - test512[i * 2] = 1; - } - - /* encode overflow */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - rc = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(rc == -1); - - t_start512 = clock(); - rc512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(rc512 == -1); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_overflow_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_overflow(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Overflow test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_range_avx512(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0, rc = 0, rc512 = 0; - int dlen = 0, dlen512 = 0; - - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); - - for (i = diff_len; i > 0; i--) { - buffer[1000 + i] = i; - test[1000 + i] = i + 4; - buffer512[1000 + i] = i; - test512[1000 + i] = i + 4; - } - - buffer[1000 + diff_len + 3] = 103; - test[1000 + diff_len + 3] = 107; - - buffer[1000 + diff_len + 5] = 105; - test[1000 + diff_len + 5] = 109; - - buffer512[1000 + diff_len + 3] = 103; - test512[1000 + diff_len + 3] = 107; - - buffer512[1000 + diff_len + 5] = 105; - test512[1000 + diff_len + 5] = 109; - - /* test encode/decode */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - rc = xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); - g_assert(rc < XBZRLE_PAGE_SIZE); - g_assert(memcmp(test, buffer, XBZRLE_PAGE_SIZE) == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - rc512 = xbzrle_decode_buffer(compressed512, dlen512, test512, XBZRLE_PAGE_SIZE); - g_assert(rc512 < XBZRLE_PAGE_SIZE); - g_assert(memcmp(test512, buffer512, XBZRLE_PAGE_SIZE) == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_range_avx512(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Encode decode test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_random(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0, rc = 0, rc512 = 0; - int dlen = 0, dlen512 = 0; - - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1); - /* store the index of diff */ - int dirty_index[diff_len]; - for (int j = 0; j < diff_len; j++) { - dirty_index[j] = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1); - } - for (i = diff_len - 1; i >= 0; i--) { - buffer[dirty_index[i]] = i; - test[dirty_index[i]] = i + 4; - buffer512[dirty_index[i]] = i; - test512[dirty_index[i]] = i + 4; - } - - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - rc = xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); - g_assert(rc < XBZRLE_PAGE_SIZE); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - rc512 = xbzrle_decode_buffer(compressed512, dlen512, test512, XBZRLE_PAGE_SIZE); - g_assert(rc512 < XBZRLE_PAGE_SIZE); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_random_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_random(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Random test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} -#endif - -int main(int argc, char **argv) -{ - g_test_init(&argc, &argv, NULL); - g_test_rand_int(); - #if defined(CONFIG_AVX512BW_OPT) - if (likely(is_cpu_support_avx512bw)) { - g_test_add_func("/xbzrle/encode_decode_zero", test_encode_decode_zero_avx512); - g_test_add_func("/xbzrle/encode_decode_unchanged", - test_encode_decode_unchanged_avx512); - g_test_add_func("/xbzrle/encode_decode_1_byte", test_encode_decode_1_byte_avx512); - g_test_add_func("/xbzrle/encode_decode_overflow", - test_encode_decode_overflow_avx512); - g_test_add_func("/xbzrle/encode_decode", test_encode_decode_avx512); - g_test_add_func("/xbzrle/encode_decode_random", test_encode_decode_random_avx512); - } - #endif - return g_test_run(); -} diff --git a/tests/unit/test-xbzrle.c b/tests/unit/test-xbzrle.c index 547046d093..b6996de69a 100644 --- a/tests/unit/test-xbzrle.c +++ b/tests/unit/test-xbzrle.c @@ -16,35 +16,6 @@ #define XBZRLE_PAGE_SIZE 4096 -int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, - uint8_t *, int) = xbzrle_encode_buffer; -#if defined(CONFIG_AVX512BW_OPT) -#include "qemu/cpuid.h" -static void __attribute__((constructor)) init_cpu_flag(void) -{ - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - if (max >= 1) { - __cpuid(1, a, b, c, d); - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - int bv; - __asm("xgetbv" : "=a"(bv), "=d"(d) : "c"(0)); - __cpuid_count(7, 0, a, b, c, d); - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512BW)) { - xbzrle_encode_buffer_func = xbzrle_encode_buffer_avx512; - } - } - } - return ; -} -#endif - static void test_uleb(void) { uint32_t i, val; @@ -83,8 +54,8 @@ static void test_encode_decode_zero(void) buffer[1000 + diff_len + 5] = 105; /* encode zero page */ - dlen = xbzrle_encode_buffer_func(buffer, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(buffer, buffer, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(dlen == 0); g_free(buffer); @@ -107,8 +78,8 @@ static void test_encode_decode_unchanged(void) test[1000 + diff_len + 5] = 109; /* test unchanged buffer */ - dlen = xbzrle_encode_buffer_func(test, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(test, test, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(dlen == 0); g_free(test); @@ -125,8 +96,8 @@ static void test_encode_decode_1_byte(void) test[XBZRLE_PAGE_SIZE - 1] = 1; - dlen = xbzrle_encode_buffer_func(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(dlen == (uleb128_encode_small(&buf[0], 4095) + 2)); rc = xbzrle_decode_buffer(compressed, dlen, buffer, XBZRLE_PAGE_SIZE); @@ -150,8 +121,8 @@ static void test_encode_decode_overflow(void) } /* encode overflow */ - rc = xbzrle_encode_buffer_func(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + rc = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(rc == -1); g_free(buffer); @@ -181,8 +152,8 @@ static void encode_decode_range(void) test[1000 + diff_len + 5] = 109; /* test encode/decode */ - dlen = xbzrle_encode_buffer_func(test, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); rc = xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); g_assert(rc < XBZRLE_PAGE_SIZE); diff --git a/tests/bench/meson.build b/tests/bench/meson.build index 4e6b469066..3c799dbd98 100644 --- a/tests/bench/meson.build +++ b/tests/bench/meson.build @@ -3,12 +3,6 @@ qht_bench = executable('qht-bench', sources: 'qht-bench.c', dependencies: [qemuutil]) -if have_system -xbzrle_bench = executable('xbzrle-bench', - sources: 'xbzrle-bench.c', - dependencies: [qemuutil,migration]) -endif - qtree_bench = executable('qtree-bench', sources: 'qtree-bench.c', dependencies: [qemuutil]) From patchwork Tue May 23 13:47:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684928 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729404wrt; Tue, 23 May 2023 06:50:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7FVenruyUl4za6K0vY55e8Kgit/BWcsHqKuhiaSBAu2o3uc54IejgHs7ou8ZIIEZhCefGQ X-Received: by 2002:a05:620a:460c:b0:75b:23a1:8334 with SMTP id br12-20020a05620a460c00b0075b23a18334mr4493378qkb.47.1684849803895; Tue, 23 May 2023 06:50:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849803; cv=none; d=google.com; s=arc-20160816; b=WteUc4iAMK8csqlnmmUJJxHYtsaDw4BOxmDudKUHgiNEmVb6Qv9aR413mvEuVTfD2P vAS+G+00j8gfjqkYOcH3cr5Y8yf6xusa4g/5qQrldmiUaKUfaF5l5MQ+Ki3y8rEpTY+O sNCoWntuwjvjasNMIi/59nu87pg9R1yyfJ0dQOr8S98AEwZDMCIcrWmX/jp+tRsVd4hY Xju31xBzI+iSAb7/wATrenZtZbLvy1uMe156CRd/SioUff/lwP0dMUqP1mbD4MxcxY10 Hajy+XoXsu74gw5XvZZgd389d5FFXAvBxxbQAztUmKNo450YYes32/uNyUzel0GEbrQU YifA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1XkUZP/KXpryW+1SxKnXlwtBsZlj1WR79DzIRZIAO0I=; b=Nl/dcnisXzzZBmAEg56AcMh7VjynOmTpDpK/5pynwExf/vYr/qJHs7entT1Ty7bjWr pgbyYcv2ufBuWoWah8HtcIfcbiUDa08rVZ/DzI3Hls3OB724s4a3mTi55qWOa+EzfJ4D O4aIoLbH8R4fWwa+aOcYDFFPaLc7rQLYfFoGByVrcX5e1kLF2C/tbqMetYaBG358J7AF NwPa2v27mzayKjf396kMRjfVUAt+KH0J30Z360+4k/sreig+x69ICSYPMUx4ibYlxIWJ mrzow6Qggc5n9IuDBNSARwMXJGKtFkdXfJpJ/nc2vR9lRY2AQpKK0WQQhcHvwndnTfOQ FEfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E71hhLmY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 8-20020a05621420c800b00619a496f1d7si5393666qve.38.2023.05.23.06.50.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:50:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E71hhLmY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMd-00006r-52; Tue, 23 May 2023 09:47:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMa-0008Nm-KW for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:44 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMY-000136-UL for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:44 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-64d2b42a8f9so4260967b3a.3 for ; Tue, 23 May 2023 06:47:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849662; x=1687441662; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1XkUZP/KXpryW+1SxKnXlwtBsZlj1WR79DzIRZIAO0I=; b=E71hhLmYQs6l6nC78VZg4/N92auIHK0Kqm0PpL5+4fbtlSrRibzvSdmg0E5ePRgVm3 AQrEJ49A1mQ5vpU40nmVCKHmLb5UCSVDh0KAO/An7W82O7ZQhJ5jjtp/J8u08ajdDkrT i5tN0ohfzPIn5Jsdcjn3MvloNWSr0nXhJHAoOM6IXkuAfhZJKhvb/7MXdHoaUpxzhsQt UXdgy8LiOW0iZk9/eEw/wHgQ53A7yWTopOZl3hbJDhJwvIdoR2/Uo/FxQH0QD5eSy3Pb 8xayY3WriNUN097aM8h0QvNdQ/nU2osbQY3R8+XHtkJJXORap/2NVIHklyo5EPo4aV/s ZNew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849662; x=1687441662; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1XkUZP/KXpryW+1SxKnXlwtBsZlj1WR79DzIRZIAO0I=; b=blwQoa/W22k+C3pEoNNff6u9LVn9VDdP+rPqmiG+Rqc8Q61emPd/6ciunRoe6CGp3H yKNrIEEgTkP4JBTyGFbkqYfqPArRhKetqeazWPNgUI1Je3FL/9BKgi3m5hSV7uaSnsYv Uss2zEWJxoAP9WgSVFP6hNV3hxdYdFOiozGIVgqjYWyfZ2/mjsKA9uJWmzDjP51so3E+ bSOXdIt1dJNWNHE6znqhs0b/cn3CTsDoOLp6X726vmtNB0nVWyt+Yp9KN9sMz5YULAFi SpZqKOr5r+RH3z7EvkpUoU0GFYjyQco6bJOJmH0WStL4+6W0jp0cX6+OpOVDP79flROr b+ow== X-Gm-Message-State: AC+VfDwXFMQ7u+4WvoqPv+L8hmtcO2rMAyxZXYlzMKe1Sa6I02F9sxlV q+PvZCmHEZldr6s0lfKDl8gdVXTV3KsnTfm0U0g= X-Received: by 2002:a05:6a00:2e98:b0:64d:3e99:83a5 with SMTP id fd24-20020a056a002e9800b0064d3e9983a5mr18368794pfb.26.1684849662209; Tue, 23 May 2023 06:47:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, Juan Quintela Subject: [PATCH v2 08/27] migration: Build migration_files once Date: Tue, 23 May 2023 06:47:14 -0700 Message-Id: <20230523134733.678646-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The items in migration_files are built for libmigration and included info softmmu_ss from there; no need to also include them directly. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- migration/meson.build | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/meson.build b/migration/meson.build index a8e01e70ae..8ba6e420fe 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -8,7 +8,6 @@ migration_files = files( 'qemu-file.c', 'yank_functions.c', ) -softmmu_ss.add(migration_files) softmmu_ss.add(files( 'block-dirty-bitmap.c', From patchwork Tue May 23 13:47:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684917 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728400wrt; Tue, 23 May 2023 06:47:56 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4wLL5iMEvyNZafCyVZ2332AlC3zvmhrpW2pxtlheB7kpSLhbeaDrASLf3UWP2mzBvdN9tw X-Received: by 2002:a05:6214:226a:b0:623:5dc2:883b with SMTP id gs10-20020a056214226a00b006235dc2883bmr29204775qvb.24.1684849676308; Tue, 23 May 2023 06:47:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849676; cv=none; d=google.com; s=arc-20160816; b=dj1BoSSTbarF5z+pWKjCTNxayjCxEYlZsgITaDiTQhmx/gwjo2wXheHb2v6+8Jud2U WksKAYLVkeY1xuQBELxhYHYQAY/WLMiB5/vutzbFo/kkWC+oJUus226mYBeyrzpviPyU pQreMHES/g0EQ/SwZC7x/vbDfESXy7XKJySpaN/fmPe0/+A+IVNGmNmjZe4MEG8aRAap 8vCU/KYok5dzxaSFYDYWOFYpWpQ+80/Thnjujqj4mVqHTM/JLWOwWMBlCgDTD8tvh542 n20olvVuh8GspeYO3z7Ae8NN1gIxR9XSvB3Pkj8W0qUt5UIEb90TXgcDjikq1baOOnYl P1bQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LBBBG0WTc1cH3EmTG+zv0uyjIvA9PNjDKH+Kc3n5h9w=; b=yI3C8Xtnk1OBYAYR0LPaZDL1l/Ghb5R91KFKPhMNuyKhRqLC+YOJmTxbP+1fEEQlhJ YtCztX/Pb0lWp2ISWcIDh9iVdHwcBnF3y+YzLHMP3AVBwulzl+MXN+hbbN6nF9zha6x+ 6cPbUmYzvdP7HpNYPrVt3iEt9Y06zs7+YxxtO6+avDqJ8G6nI8QlQEGIH66uu+EooOiC PyHyRT9LeYlP0KaDCZjFYCLpmQRKhAXPMpbw1+tJJBC/JgQxirXntQlVon9w64PK6kaA /kOaALJmtBLr0WrtPQA5Mfk+VHqKs3E00ETcK0nJ2fCGXSzKPsOzugbTQYHwQTQlC1qd 68KA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=sPRlxV6b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id if13-20020a0562141c4d00b006238b0d1487si1408981qvb.541.2023.05.23.06.47.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:47:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=sPRlxV6b; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMe-0000FQ-Qn; Tue, 23 May 2023 09:47:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMc-0008WK-Lh for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:46 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMa-000150-HW for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:46 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64d604cc0aaso2237340b3a.2 for ; Tue, 23 May 2023 06:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849663; x=1687441663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LBBBG0WTc1cH3EmTG+zv0uyjIvA9PNjDKH+Kc3n5h9w=; b=sPRlxV6bYzqo1WAGLxv6o9jUohqZYxV7w8ezSXtuf3ODgXS8hzLYZ2PvOnubO3I0el 67/caqvlsdwhy8jst/pdVQkefV89IcP4prN9F0lyvyAoxHM7AukMxNVrSm1H4EtXOkVV +wNz0sIFMsX522LgLUfjEl0t8o6HP84cElFGAUoNMKz7o3HEcOhJ+zkL3zCgBz/kQWlZ 6VxpAuQFGU088C+lcxHlhsDtaTHXUgNjpiX47pspkSACaK5A8z9szt3jYzfM2LBDEQmM e+xyBpslnBhxb871HUIb0jKFQ/RroZqgBcU274QmdPQVDORsEOCI/sOzvwD0JIBxIKZ/ fLxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849663; x=1687441663; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LBBBG0WTc1cH3EmTG+zv0uyjIvA9PNjDKH+Kc3n5h9w=; b=gV5p2GnprEEm9D0zwinKQwdF/5hfqpWdf6s225O1tHARwL4xcGOyQn6k3Y5kVyrBTF bM/6AdB0n2+RUg7wQNUKl+nqbRk9dTGTh6YFdDCb3l8BKl8pLA/xZp4KKOBf2vm8K6jh ZCjAff+V6XNNrkXBdCRfRY6c51gz+8O7DTOxJdfaJwDZExvheAT3h6iAWLAih08bnm4t eNYYJeSpBCz4s8mH4KAizaB2DaxV+wKwiKJAMF2LlBJDQ9ryjMVzNkAPyp9BoA10jINu BqjMLF/iLRbRsxPLYEaJJWIVxHQrQKUmDnp0EVB2s+ydl/IXtsdqTnrMgJh5xs4wGrjv 7mcA== X-Gm-Message-State: AC+VfDy+Zn/pYem8UVsf+b0BqkUiB0IK7D/amMLEPQbWEQl2l8f46trB No6rytEt3HvzV93UNoOoty08IezCrynqxWYfLl4= X-Received: by 2002:a05:6a21:338f:b0:10a:f3df:b86e with SMTP id yy15-20020a056a21338f00b0010af3dfb86emr10034639pzb.44.1684849662995; Tue, 23 May 2023 06:47:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 09/27] util: Add cpuinfo-aarch64.c Date: Tue, 23 May 2023 06:47:15 -0700 Message-Id: <20230523134733.678646-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Move the code from tcg/. The only use of these bits so far is with respect to the atomicity of tcg operations. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/aarch64/host/cpuinfo.h | 22 ++++++++++ tcg/aarch64/tcg-target.h | 6 ++- util/cpuinfo-aarch64.c | 67 +++++++++++++++++++++++++++++ tcg/aarch64/tcg-target.c.inc | 40 ----------------- util/meson.build | 4 +- 5 files changed, 96 insertions(+), 43 deletions(-) create mode 100644 host/include/aarch64/host/cpuinfo.h create mode 100644 util/cpuinfo-aarch64.c diff --git a/host/include/aarch64/host/cpuinfo.h b/host/include/aarch64/host/cpuinfo.h new file mode 100644 index 0000000000..82227890b4 --- /dev/null +++ b/host/include/aarch64/host/cpuinfo.h @@ -0,0 +1,22 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for AArch64. + */ + +#ifndef HOST_CPUINFO_H +#define HOST_CPUINFO_H + +#define CPUINFO_ALWAYS (1u << 0) /* so cpuinfo is nonzero */ +#define CPUINFO_LSE (1u << 1) +#define CPUINFO_LSE2 (1u << 2) + +/* Initialized with a constructor. */ +extern unsigned cpuinfo; + +/* + * We cannot rely on constructor ordering, so other constructors must + * use the function interface rather than the variable above. + */ +unsigned cpuinfo_init(void); + +#endif /* HOST_CPUINFO_H */ diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 74ee2ed255..d5f7614880 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -13,6 +13,8 @@ #ifndef AARCH64_TCG_TARGET_H #define AARCH64_TCG_TARGET_H +#include "host/cpuinfo.h" + #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24 #define MAX_CODE_GEN_BUFFER_SIZE ((size_t)-1) @@ -57,8 +59,8 @@ typedef enum { #define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN #define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL -extern bool have_lse; -extern bool have_lse2; +#define have_lse (cpuinfo & CPUINFO_LSE) +#define have_lse2 (cpuinfo & CPUINFO_LSE2) /* optional instructions */ #define TCG_TARGET_HAS_div_i32 1 diff --git a/util/cpuinfo-aarch64.c b/util/cpuinfo-aarch64.c new file mode 100644 index 0000000000..f99acb7884 --- /dev/null +++ b/util/cpuinfo-aarch64.c @@ -0,0 +1,67 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for AArch64. + */ + +#include "qemu/osdep.h" +#include "host/cpuinfo.h" + +#ifdef CONFIG_LINUX +# ifdef CONFIG_GETAUXVAL +# include +# else +# include +# include "elf.h" +# endif +#endif +#ifdef CONFIG_DARWIN +# include +#endif + +unsigned cpuinfo; + +#ifdef CONFIG_DARWIN +static bool sysctl_for_bool(const char *name) +{ + int val = 0; + size_t len = sizeof(val); + + if (sysctlbyname(name, &val, &len, NULL, 0) == 0) { + return val != 0; + } + + /* + * We might in the future ask for properties not present in older kernels, + * but we're only asking about static properties, all of which should be + * 'int'. So we shouln't see ENOMEM (val too small), or any of the other + * more exotic errors. + */ + assert(errno == ENOENT); + return false; +} +#endif + +/* Called both as constructor and (possibly) via other constructors. */ +unsigned __attribute__((constructor)) cpuinfo_init(void) +{ + unsigned info = cpuinfo; + + if (info) { + return info; + } + + info = CPUINFO_ALWAYS; + +#ifdef CONFIG_LINUX + unsigned long hwcap = qemu_getauxval(AT_HWCAP); + info |= (hwcap & HWCAP_ATOMICS ? CPUINFO_LSE : 0); + info |= (hwcap & HWCAP_USCAT ? CPUINFO_LSE2 : 0); +#endif +#ifdef CONFIG_DARWIN + info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE") * CPUINFO_LSE; + info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE2") * CPUINFO_LSE2; +#endif + + cpuinfo = info; + return info; +} diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index bc6b99a1bd..84283665e7 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -13,12 +13,6 @@ #include "../tcg-ldst.c.inc" #include "../tcg-pool.c.inc" #include "qemu/bitops.h" -#ifdef __linux__ -#include -#endif -#ifdef CONFIG_DARWIN -#include -#endif /* We're going to re-use TCGType in setting of the SF bit, which controls the size of the operation performed. If we know the values match, it @@ -77,9 +71,6 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) return TCG_REG_X0 + slot; } -bool have_lse; -bool have_lse2; - #define TCG_REG_TMP TCG_REG_X30 #define TCG_VEC_TMP TCG_REG_V31 @@ -2878,39 +2869,8 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) } } -#ifdef CONFIG_DARWIN -static bool sysctl_for_bool(const char *name) -{ - int val = 0; - size_t len = sizeof(val); - - if (sysctlbyname(name, &val, &len, NULL, 0) == 0) { - return val != 0; - } - - /* - * We might in the future ask for properties not present in older kernels, - * but we're only asking about static properties, all of which should be - * 'int'. So we shouln't see ENOMEM (val too small), or any of the other - * more exotic errors. - */ - assert(errno == ENOENT); - return false; -} -#endif - static void tcg_target_init(TCGContext *s) { -#ifdef __linux__ - unsigned long hwcap = qemu_getauxval(AT_HWCAP); - have_lse = hwcap & HWCAP_ATOMICS; - have_lse2 = hwcap & HWCAP_USCAT; -#endif -#ifdef CONFIG_DARWIN - have_lse = sysctl_for_bool("hw.optional.arm.FEAT_LSE"); - have_lse2 = sysctl_for_bool("hw.optional.arm.FEAT_LSE2"); -#endif - tcg_target_available_regs[TCG_TYPE_I32] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_I64] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_V64] = 0xffffffff00000000ull; diff --git a/util/meson.build b/util/meson.build index b3be9fad5d..3a93071d27 100644 --- a/util/meson.build +++ b/util/meson.build @@ -109,6 +109,8 @@ if have_block util_ss.add(when: 'CONFIG_LINUX', if_true: files('vfio-helpers.c')) endif -if cpu in ['x86', 'x86_64'] +if cpu == 'aarch64' + util_ss.add(files('cpuinfo-aarch64.c')) +elif cpu in ['x86', 'x86_64'] util_ss.add(files('cpuinfo-i386.c')) endif From patchwork Tue May 23 13:47:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684940 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730649wrt; Tue, 23 May 2023 06:52:55 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7yH2ATwCTcn6hEzkiBlOG8FAPh8BUdxMj08PA9vCUGAyiburQsoFwS+c8aDeaPusyuVf7e X-Received: by 2002:ad4:5f08:0:b0:5e6:1bf5:1ae0 with SMTP id fo8-20020ad45f08000000b005e61bf51ae0mr25099226qvb.18.1684849975019; Tue, 23 May 2023 06:52:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849974; cv=none; d=google.com; s=arc-20160816; b=z4f5nht9ACQV3LtbuED7uoNVxOEl6ynOPxj4FluEINVjPeeW9DM1/1dQPZmvJhDOHG KVdeRO5CII3YCYJLY+x9ZQiFDxPQ6VsaLTju+8Rv7WDMeVKnwoH3G1s5lqx6X815SEZ1 Q0pP0YtQvEQkH8J0ni3eJfaE6+NDDmYkyZQ0M5jsOk1cRBIrOc/RBousK6aGhnm87Wdw uKSZuP+GTOXJSn67X7dr09Y6zJ9rTHDULzihxO9g022PQjYHkytrSrcpqI1v7V70qnKc yKF3iMFypjGFeOpn/lCHB8wK66bYxdDsfa9XQcK1U1Wb/1wofyM3ljE/+0rKqUVqWELu keXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Sfq7XNVW+TNC8m0ClxVe8Ju7gjJ127s2YkvolyTgAE8=; b=a+3usWrfcfH1zHeciR43hBL3+/1UkzdPT0SFHLGJG38BmdVR3lOGemBolEoCR8chmX Ucf2zrskWnsBOqrjB8PWhSBBcAmEzUfnMrAP1ZY/pKMwdC7hE9LRe5SKIUgDY7EvlpAm ChpDkp3DWptY3h0vEvYmPAuTbxTUOcxXmgw1oid3t2FypJI5iRpg1XEB2a33Y0ZrMXiv 0o5NxehXQjgrrEU68hhKHXF3hyoKfBfjIz6T+3GDUe7xm95e5KkhDtct0yF5D9hU70H+ iFQlI8VXXDYgtO5dnAYjN2mbf/tmQaiIbyonW6eK8c6ZgR54tbrdi9pP0Oto2Xlav1sV JFrg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=o7qCgS+o; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 12-20020a05621420ac00b005ef5b0e47efsi5017184qvd.125.2023.05.23.06.52.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:52:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=o7qCgS+o; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMh-0000RD-3G; Tue, 23 May 2023 09:47:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMe-0000Ct-7c for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:48 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMc-00016H-4O for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:47 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64d5b4c400fso3385977b3a.1 for ; Tue, 23 May 2023 06:47:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849664; x=1687441664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Sfq7XNVW+TNC8m0ClxVe8Ju7gjJ127s2YkvolyTgAE8=; b=o7qCgS+o4H4Y9YZKxceTnFnh2EsHtJsqhHJbGNi/dJ0ZYy5KV5+79AuRo5Guh+o8ug 39Dfs8ndg5zJ6RGwPK0sS79qfGoKUlZZrLF0SudJ8AlF2prIeSt1ihrl82zG1685ZK5V yNuNAeHWZPK4ZiPWrXMb89gEi2HRiSDOAOwugL0I2m8+g+ljfm4aP/cv78EIPCIFyDhG E2V5mfjPPxw20d0960rsnvAmEBnKGtsA2Nze3Dx7MsQZM9HLdznIdEtF6tuWrvd7jw9/ 7QtKcVLt922PCtWt8NrIEbwC3wqwyVdHpqNO0uSHtwyMFgGzqAOyA8a5dWWYaY8BlaHG KR3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849664; x=1687441664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sfq7XNVW+TNC8m0ClxVe8Ju7gjJ127s2YkvolyTgAE8=; b=H+qn385mgUUraJ3XR/EBkMl6lzxvKViXrbh3v9T5rmxHV3JKjEbeTEuiJABOjHKKqd SqpV7bq+tYyUXHJoV7Sq39K7c6wkLqt3zitk7h1rw5rkaxSbMhXnvFrtVZMHX3XHktTv FSz02LIAKcRlfSeOllfavjYhMDa8vKup//jG9HnOn5ZdmnGIPjAdMiciz7s9h/x0Vzbg x4tdm/k9ayxPbhAu0KCG2SQ/NKsXdzSDSn6DKciXIhWAb0O0I+QFU6GKYzEflXa0ErzT FukTNpPH/i8frB82C1R3xsfgpUsppG3ZLaACKFbYHqFjTCKny2CgOyks0e8uTzlMwp8X fRxA== X-Gm-Message-State: AC+VfDwC2U7DZ68EDfJWnhAWndSByIVVUoOHuKa72Xauf4aixAANYb3I NSMA7tHJUkBRmfU0uitOQNoKKbzOxdr7lSi4SB4= X-Received: by 2002:a05:6a00:2405:b0:640:f313:efba with SMTP id z5-20020a056a00240500b00640f313efbamr16790591pfh.19.1684849663685; Tue, 23 May 2023 06:47:43 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 10/27] include/host: Split out atomic128-cas.h Date: Tue, 23 May 2023 06:47:16 -0700 Message-Id: <20230523134733.678646-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Separates the aarch64-specific portion into its own file. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/aarch64/host/atomic128-cas.h | 43 ++++++++++++++++++ host/include/generic/host/atomic128-cas.h | 43 ++++++++++++++++++ include/qemu/atomic128.h | 55 +---------------------- 3 files changed, 87 insertions(+), 54 deletions(-) create mode 100644 host/include/aarch64/host/atomic128-cas.h create mode 100644 host/include/generic/host/atomic128-cas.h diff --git a/host/include/aarch64/host/atomic128-cas.h b/host/include/aarch64/host/atomic128-cas.h new file mode 100644 index 0000000000..80de58e06d --- /dev/null +++ b/host/include/aarch64/host/atomic128-cas.h @@ -0,0 +1,43 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Compare-and-swap for 128-bit atomic operations, AArch64 version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_CAS_H +#define AARCH64_ATOMIC128_CAS_H + +/* Through gcc 10, aarch64 has no support for 128-bit atomics. */ +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) +#include "host/include/generic/host/atomic128-cas.h" +#else +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + uint64_t cmpl = int128_getlo(cmp), cmph = int128_gethi(cmp); + uint64_t newl = int128_getlo(new), newh = int128_gethi(new); + uint64_t oldl, oldh; + uint32_t tmp; + + asm("0: ldaxp %[oldl], %[oldh], %[mem]\n\t" + "cmp %[oldl], %[cmpl]\n\t" + "ccmp %[oldh], %[cmph], #0, eq\n\t" + "b.ne 1f\n\t" + "stlxp %w[tmp], %[newl], %[newh], %[mem]\n\t" + "cbnz %w[tmp], 0b\n" + "1:" + : [mem] "+m"(*ptr), [tmp] "=&r"(tmp), + [oldl] "=&r"(oldl), [oldh] "=&r"(oldh) + : [cmpl] "r"(cmpl), [cmph] "r"(cmph), + [newl] "r"(newl), [newh] "r"(newh) + : "memory", "cc"); + + return int128_make128(oldl, oldh); +} +# define HAVE_CMPXCHG128 1 +#endif + +#endif /* AARCH64_ATOMIC128_CAS_H */ diff --git a/host/include/generic/host/atomic128-cas.h b/host/include/generic/host/atomic128-cas.h new file mode 100644 index 0000000000..513622fe34 --- /dev/null +++ b/host/include/generic/host/atomic128-cas.h @@ -0,0 +1,43 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Compare-and-swap for 128-bit atomic operations, generic version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef HOST_ATOMIC128_CAS_H +#define HOST_ATOMIC128_CAS_H + +#if defined(CONFIG_ATOMIC128) +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + Int128Alias r, c, n; + + c.s = cmp; + n.s = new; + r.i = qatomic_cmpxchg__nocheck((__int128_t *)ptr, c.i, n.i); + return r.s; +} +# define HAVE_CMPXCHG128 1 +#elif defined(CONFIG_CMPXCHG128) +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + Int128Alias r, c, n; + + c.s = cmp; + n.s = new; + r.i = __sync_val_compare_and_swap_16((__int128_t *)ptr, c.i, n.i); + return r.s; +} +# define HAVE_CMPXCHG128 1 +#else +/* Fallback definition that must be optimized away, or error. */ +Int128 QEMU_ERROR("unsupported atomic") + atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new); +# define HAVE_CMPXCHG128 0 +#endif + +#endif /* HOST_ATOMIC128_CAS_H */ diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h index d0ba0b9c65..10a2322c44 100644 --- a/include/qemu/atomic128.h +++ b/include/qemu/atomic128.h @@ -41,60 +41,7 @@ * Therefore, special case each platform. */ -#if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) -{ - Int128Alias r, c, n; - - c.s = cmp; - n.s = new; - r.i = qatomic_cmpxchg__nocheck((__int128_t *)ptr, c.i, n.i); - return r.s; -} -# define HAVE_CMPXCHG128 1 -#elif defined(CONFIG_CMPXCHG128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) -{ - Int128Alias r, c, n; - - c.s = cmp; - n.s = new; - r.i = __sync_val_compare_and_swap_16((__int128_t *)ptr, c.i, n.i); - return r.s; -} -# define HAVE_CMPXCHG128 1 -#elif defined(__aarch64__) -/* Through gcc 8, aarch64 has no support for 128-bit at all. */ -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) -{ - uint64_t cmpl = int128_getlo(cmp), cmph = int128_gethi(cmp); - uint64_t newl = int128_getlo(new), newh = int128_gethi(new); - uint64_t oldl, oldh; - uint32_t tmp; - - asm("0: ldaxp %[oldl], %[oldh], %[mem]\n\t" - "cmp %[oldl], %[cmpl]\n\t" - "ccmp %[oldh], %[cmph], #0, eq\n\t" - "b.ne 1f\n\t" - "stlxp %w[tmp], %[newl], %[newh], %[mem]\n\t" - "cbnz %w[tmp], 0b\n" - "1:" - : [mem] "+m"(*ptr), [tmp] "=&r"(tmp), - [oldl] "=&r"(oldl), [oldh] "=&r"(oldh) - : [cmpl] "r"(cmpl), [cmph] "r"(cmph), - [newl] "r"(newl), [newh] "r"(newh) - : "memory", "cc"); - - return int128_make128(oldl, oldh); -} -# define HAVE_CMPXCHG128 1 -#else -/* Fallback definition that must be optimized away, or error. */ -Int128 QEMU_ERROR("unsupported atomic") - atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new); -# define HAVE_CMPXCHG128 0 -#endif /* Some definition for HAVE_CMPXCHG128 */ - +#include "host/atomic128-cas.h" #if defined(CONFIG_ATOMIC128) static inline Int128 atomic16_read(Int128 *ptr) From patchwork Tue May 23 13:47:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684941 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730686wrt; Tue, 23 May 2023 06:53:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7qYG4NfkeahBdxtZ8p9r392K8wzG/7sD+fkk9Obq0ZsM3QPNGu6D255twOO/lNmoSgzkF+ X-Received: by 2002:a05:622a:352:b0:3f6:beb9:c856 with SMTP id r18-20020a05622a035200b003f6beb9c856mr1080040qtw.2.1684849980029; Tue, 23 May 2023 06:53:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849980; cv=none; d=google.com; s=arc-20160816; b=Jfqtj6rIP+WY1QzL2mOXFuSJhj6KflH0OV/AebRyJgNo06cCvcf8w9e0rj0wSZcdyF lXEgn1QODWV+Q9dNO+KsDhGJlBZz7TU++Ylk9cG/Wo5fHB+dDV7YMCl1H1WvfprQ8Jlh igbH22xYSTT+t+bvGnwxuyxYVLLnN0hKWlWYcOIFFpMDhUjGRnrJVnORdJCpFK3BMZ99 6hyskH6r9gyirAsWhIKevhTvKj9foMwLELPunRd+iEsLM8znL+vQuEzDtvJW5q/gNcPX nApoSgB7BUvd00ye7ID19ouZzrZGsePMNPamsNXUGaPrx0UyPKgtdSJ4xrriDgrDlpIi tvrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=jB5qzpA06Oq7YWao0ePdlph12dZD3cppuuTihIbPb0c=; b=pH5CTsAPkLraEz0wK974WUKjNfI5YSAH2d1x/t4or9Nyjj5RQOujdM5SnMfow+qPh4 mrpJXWuDBag2kGOld7q/LIeR811I1xksFFAUvy7FG6sLvDHS4dZ9NitJcyZDCvvsjmk7 gkSDLDhYBEqEoaxmWpokWU9z4vu3vqr0sMU46hq9lGcYiAE82fSm/B5yzG0AYiLiU15B xsq33Lutfc5stSLH1r2ocpeFxaNeB/jylpQZ9f4CCWe8VfDJSyZcIP+AZDaidpMpN/JK UzRJXNfRGNwesQp+ha5xRsvVY/KV/mpraUOnCs0ynVtbT6XFx6G+SerJDbnO7WZC7HPQ UvLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="LWB/UA5R"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n7-20020a05622a040700b003e4e39b3567si5025253qtx.593.2023.05.23.06.52.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:53:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="LWB/UA5R"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMf-0000HN-IM; Tue, 23 May 2023 09:47:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMe-0000CZ-6K for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:48 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMc-00016c-2G for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:47 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-64d5b4c3ffeso2141788b3a.2 for ; Tue, 23 May 2023 06:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849664; x=1687441664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jB5qzpA06Oq7YWao0ePdlph12dZD3cppuuTihIbPb0c=; b=LWB/UA5R8n+T+In/0ithAEutCj17Ka5SLgsu++S/ImQLNF85rCRy3TiLRf0wADaYSH M1IKv2w3TVHnriRrNeuFwByro060PfIPHyOE3jdFYB/pINruokNIR9HGVdFo3iYuTfX9 KW8j+Hln2HUJwayTdHxsaWflXvArRxsMg7YjCcUAnmv9r8Q06T0v2sf/WYaQqrJx29Sp AAHIrPmtpSPIPJ81X+rj/b1aA4K+LfCfrp9mYvN0eI5xabQbfNYzu8F4kANoh0kSk+J4 KOk4fmCCV4gerCWlfaIoNcNm325GmxIL1gt+yb5ByEwi9OJXU6jZiD4ys5CjKyz/fhNA STtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849664; x=1687441664; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jB5qzpA06Oq7YWao0ePdlph12dZD3cppuuTihIbPb0c=; b=KpGF+Tf1DzvMRPGloce9K6beVBvzs+TpzQP0m365tbj0OlQFVgB5/NajE4OWSgDn2Y wv1Ouwf+2wdmW9WcUIGNQQyyIhuxrO7FvPFImAPwJ3zkmwtvCQxV4rHaPf+l0DTbh2LA npk2AYqOrDZT7Vivhf8v2gyA1SaSFeWTnChEW9xRVxQ3aVt0wqS4ppOQ/vIWF/zOwJIM nNLSl7YMqKlqdmMFiToCL3WgAPL8Vp3l6DD6WbMSPhS1xCDGZzziH7fm5FII2EnC8WNZ GGfzg9UrHPZQykdZ5Nq6x/e3dSrQ2q0GabKCjW9QHEBk682JCmmWtiDUEI6nHOH8nUVz v4/A== X-Gm-Message-State: AC+VfDxjZhrTCxukAVK419hKvivw5a+q5LCJg2e7DeBVOVpv72Fmbxdr yfAsKpg2Q7pf2hhHVgVVDzKlNvFL8ASbuS118Jw= X-Received: by 2002:a05:6a00:1948:b0:640:defd:a6d5 with SMTP id s8-20020a056a00194800b00640defda6d5mr15899311pfk.12.1684849664441; Tue, 23 May 2023 06:47:44 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 11/27] include/host: Split out atomic128-ldst.h Date: Tue, 23 May 2023 06:47:17 -0700 Message-Id: <20230523134733.678646-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Separates the aarch64-specific portion into its own file. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/aarch64/host/atomic128-ldst.h | 49 ++++++++++++++ host/include/generic/host/atomic128-ldst.h | 57 +++++++++++++++++ include/qemu/atomic128.h | 74 +--------------------- 3 files changed, 107 insertions(+), 73 deletions(-) create mode 100644 host/include/aarch64/host/atomic128-ldst.h create mode 100644 host/include/generic/host/atomic128-ldst.h diff --git a/host/include/aarch64/host/atomic128-ldst.h b/host/include/aarch64/host/atomic128-ldst.h new file mode 100644 index 0000000000..bd61fce50d --- /dev/null +++ b/host/include/aarch64/host/atomic128-ldst.h @@ -0,0 +1,49 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, AArch64 version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_LDST_H +#define AARCH64_ATOMIC128_LDST_H + +/* Through gcc 10, aarch64 has no support for 128-bit atomics. */ +#if !defined(CONFIG_ATOMIC128) && !defined(CONFIG_USER_ONLY) +/* We can do better than cmpxchg for AArch64. */ +static inline Int128 atomic16_read(Int128 *ptr) +{ + uint64_t l, h; + uint32_t tmp; + + /* The load must be paired with the store to guarantee not tearing. */ + asm("0: ldxp %[l], %[h], %[mem]\n\t" + "stxp %w[tmp], %[l], %[h], %[mem]\n\t" + "cbnz %w[tmp], 0b" + : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + + return int128_make128(l, h); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + uint64_t l = int128_getlo(val), h = int128_gethi(val); + uint64_t t1, t2; + + /* Load into temporaries to acquire the exclusive access lock. */ + asm("0: ldxp %[t1], %[t2], %[mem]\n\t" + "stxp %w[t1], %[l], %[h], %[mem]\n\t" + "cbnz %w[t1], 0b" + : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) + : [l] "r"(l), [h] "r"(h)); +} + +# define HAVE_ATOMIC128 1 +#else +#include "host/include/generic/host/atomic128-ldst.h" +#endif + +#endif /* AARCH64_ATOMIC128_LDST_H */ diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h new file mode 100644 index 0000000000..e7354a9255 --- /dev/null +++ b/host/include/generic/host/atomic128-ldst.h @@ -0,0 +1,57 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, generic version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef HOST_ATOMIC128_LDST_H +#define HOST_ATOMIC128_LDST_H + +#if defined(CONFIG_ATOMIC128) +static inline Int128 atomic16_read(Int128 *ptr) +{ + Int128Alias r; + + r.i = qatomic_read__nocheck((__int128_t *)ptr); + return r.s; +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + Int128Alias v; + + v.s = val; + qatomic_set__nocheck((__int128_t *)ptr, v.i); +} + +# define HAVE_ATOMIC128 1 +#elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 +static inline Int128 atomic16_read(Int128 *ptr) +{ + /* Maybe replace 0 with 0, returning the old value. */ + Int128 z = int128_make64(0); + return atomic16_cmpxchg(ptr, z, z); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + Int128 old = *ptr, cmp; + do { + cmp = old; + old = atomic16_cmpxchg(ptr, cmp, val); + } while (int128_ne(old, cmp)); +} + +# define HAVE_ATOMIC128 1 +#else +/* Fallback definitions that must be optimized away, or error. */ +Int128 QEMU_ERROR("unsupported atomic") atomic16_read(Int128 *ptr); +void QEMU_ERROR("unsupported atomic") atomic16_set(Int128 *ptr, Int128 val); +# define HAVE_ATOMIC128 0 +#endif + +#endif /* HOST_ATOMIC128_LDST_H */ diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h index 10a2322c44..3a8adb4d47 100644 --- a/include/qemu/atomic128.h +++ b/include/qemu/atomic128.h @@ -42,78 +42,6 @@ */ #include "host/atomic128-cas.h" - -#if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_read(Int128 *ptr) -{ - Int128Alias r; - - r.i = qatomic_read__nocheck((__int128_t *)ptr); - return r.s; -} - -static inline void atomic16_set(Int128 *ptr, Int128 val) -{ - Int128Alias v; - - v.s = val; - qatomic_set__nocheck((__int128_t *)ptr, v.i); -} - -# define HAVE_ATOMIC128 1 -#elif !defined(CONFIG_USER_ONLY) && defined(__aarch64__) -/* We can do better than cmpxchg for AArch64. */ -static inline Int128 atomic16_read(Int128 *ptr) -{ - uint64_t l, h; - uint32_t tmp; - - /* The load must be paired with the store to guarantee not tearing. */ - asm("0: ldxp %[l], %[h], %[mem]\n\t" - "stxp %w[tmp], %[l], %[h], %[mem]\n\t" - "cbnz %w[tmp], 0b" - : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); - - return int128_make128(l, h); -} - -static inline void atomic16_set(Int128 *ptr, Int128 val) -{ - uint64_t l = int128_getlo(val), h = int128_gethi(val); - uint64_t t1, t2; - - /* Load into temporaries to acquire the exclusive access lock. */ - asm("0: ldxp %[t1], %[t2], %[mem]\n\t" - "stxp %w[t1], %[l], %[h], %[mem]\n\t" - "cbnz %w[t1], 0b" - : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) - : [l] "r"(l), [h] "r"(h)); -} - -# define HAVE_ATOMIC128 1 -#elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 -static inline Int128 atomic16_read(Int128 *ptr) -{ - /* Maybe replace 0 with 0, returning the old value. */ - Int128 z = int128_make64(0); - return atomic16_cmpxchg(ptr, z, z); -} - -static inline void atomic16_set(Int128 *ptr, Int128 val) -{ - Int128 old = *ptr, cmp; - do { - cmp = old; - old = atomic16_cmpxchg(ptr, cmp, val); - } while (int128_ne(old, cmp)); -} - -# define HAVE_ATOMIC128 1 -#else -/* Fallback definitions that must be optimized away, or error. */ -Int128 QEMU_ERROR("unsupported atomic") atomic16_read(Int128 *ptr); -void QEMU_ERROR("unsupported atomic") atomic16_set(Int128 *ptr, Int128 val); -# define HAVE_ATOMIC128 0 -#endif /* Some definition for HAVE_ATOMIC128 */ +#include "host/atomic128-ldst.h" #endif /* QEMU_ATOMIC128_H */ From patchwork Tue May 23 13:47:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684919 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728522wrt; Tue, 23 May 2023 06:48:13 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ68MRWIcpzie3hoEhrvqmAr1MKq9ZoT9nFAkr0MQf8vgHtiw49uHa3y4JBKov78UZJ9hM+T X-Received: by 2002:a05:622a:1a2a:b0:3f6:b5b3:b7c0 with SMTP id f42-20020a05622a1a2a00b003f6b5b3b7c0mr5554431qtb.37.1684849693640; Tue, 23 May 2023 06:48:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849693; cv=none; d=google.com; s=arc-20160816; b=d6qw/8T/FxfyE82EPWgXB4utsWBf0FgPKfeJBbLE16lnf29CqpgTzXz/0Y0FHEf4M7 RRXWAgo/o0Cz2t8fun27Y0ZSlLyvk8mvRicRWfMnoSkgecvo+sq7i2dyn+jhPQZKnsz3 mryt//o2ixoO5dqXxUnl2K04m6wfCHwDRQl4sLQOC0QHLU6F+DV699Pvou8Jj5JrsM3h 34B84ToqTe+w+yyz+EjhmulmoPvIs0/kNscY2VG62HoF7JIdm3Fkyq7yTtDXcTBgxQwu U0Y8UR4swA82pvaurvlO6LPCyGswzIpWl3YUPpUKL6tC/zKNTniy+tLSxDbhOi7A5xyJ pWPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Ga9z9BMWvgZJWDH0KgOQhGEvMXXMnmqA3xWrk5W7Afk=; b=dOYx/ETBXXTDwZzZgRG39KIiewziX27vv8n5r3SwsFGazkjyrqf1qa/Y9Yni2mX4HH YkQYM2QxlLFZXCSUuyukxvtIJarZ9utatTlvvN+MNo+MDogqUNMcfHxhW6AnFXZIXzOM hpOLBlm5ZEHCszpqD2zT72cqkp2NLM+YSd/8ArdkB6tpMFLAHzus4fk/rQ1nTdZ2aYmN 7e/u/oOfJ5vWT+7yTlBlXFZK2tBnBvNXLDyYlj7IoRPtFe19XJ9ttkD52QtZrxkRlWSu 5wJs3pOF7UnOvAJrK3zJiiQQv1C9FRLuKS75hwrdI6hobCjnD48Z434C0fB9txFQzCtD 3Wjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L4hvI9bh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m8-20020a05622a118800b003f5269a2005si5222743qtk.282.2023.05.23.06.48.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L4hvI9bh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMk-0000YZ-TF; Tue, 23 May 2023 09:47:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMe-0000FS-V1 for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:48 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMd-00016k-G8 for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:48 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-64d604cc0aaso2237385b3a.2 for ; Tue, 23 May 2023 06:47:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849665; x=1687441665; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ga9z9BMWvgZJWDH0KgOQhGEvMXXMnmqA3xWrk5W7Afk=; b=L4hvI9bhaV/6tUzg9U3xkZcxFjslbo3GBhhHVrD56KYji0jPgFjhNknet2NUhLzIGG 3qimuT8aZ78lzWCIvEOxjs9555K3oIOU+lQwU5DnhjWFVGvzc78i+P9qWpNX0sxS4lQ3 ZreNyNzDksnu5d5h94fkiB3Da0GEVBSU/QoJH/gP6V0y3/lJCfqwbrbU4x6sOnhBHUep m9hw/gxjXomiafRlDr7235hqQ1ELLaH2uLs7yGjc9WRJ2cuaOBNxmv0jgfeKZ8gG7Q/n N5gYWDr267u+L9S8Agr4cbSvgXj87bqiUY+ur+TB6D+GQVqXvY7EHumqLJC67jfHXWHQ BRwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849665; x=1687441665; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ga9z9BMWvgZJWDH0KgOQhGEvMXXMnmqA3xWrk5W7Afk=; b=gv2GnM++3K2D1cHa8VZvKJEoxoFuSo8GYiVhHqDJETo1TfmDNPC9tPkiOsYauvaKAU pMxD63uaLug05OyC5m2kvFzlBan87Fsok/sEhoyS67ZVxYGfG0zDLg7dBlMidE5dddZd ZAI2NVRisQJb/4c+mwGipJdw9gaqswviac3qQAw4Xd6dVsCFm1OMVz7ERbXxWXiGedzC 7VvfyDFVLDcu9rGRLE3yqZh2BDgdBJqGB+4fpgXBFxMG1nJTPLVGeo/1S5IOi9e+RjB5 tx5TnK95ZGPmNdK+qpmjDCym3jXNep0htCFycwXAx2E/J5sXgkWStMNAsJnJKHLcknO3 2SLw== X-Gm-Message-State: AC+VfDzrcAM9z7zn71ttmGMTFNS0O772tXM0IxQtbMNV8FVbacXt6lxJ GxZ8euzQktvtA3PUAjGf8TCSZHHi37sp8Gm/c4o= X-Received: by 2002:a05:6a20:3d07:b0:10b:1c98:59b2 with SMTP id y7-20020a056a203d0700b0010b1c9859b2mr10270054pzi.21.1684849665336; Tue, 23 May 2023 06:47:45 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v2 12/27] meson: Fix detect atomic128 support with optimization Date: Tue, 23 May 2023 06:47:18 -0700 Message-Id: <20230523134733.678646-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Silly typo: sizeof(16) != 16. Fixes: e61f1efeb730 ("meson: Detect atomic128 support with optimization") Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meson.build b/meson.build index c516b911d9..ef181ff2df 100644 --- a/meson.build +++ b/meson.build @@ -2557,7 +2557,7 @@ if has_int128 # __alignof(unsigned __int128) for the host. atomic_test_128 = ''' int main(int ac, char **av) { - unsigned __int128 *p = __builtin_assume_aligned(av[ac - 1], sizeof(16)); + unsigned __int128 *p = __builtin_assume_aligned(av[ac - 1], 16); p[1] = __atomic_load_n(&p[0], __ATOMIC_RELAXED); __atomic_store_n(&p[2], p[3], __ATOMIC_RELAXED); __atomic_compare_exchange_n(&p[4], &p[5], p[6], 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED); From patchwork Tue May 23 13:47:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684942 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730844wrt; Tue, 23 May 2023 06:53:15 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5prW1AwvtECw/gKMabAtqb2bGNKCB5K359oeDgQpO70F8z3/XL7/w+G/OVGsMXEX2319SE X-Received: by 2002:a05:620a:808c:b0:75b:23a0:d9e9 with SMTP id ef12-20020a05620a808c00b0075b23a0d9e9mr4724714qkb.63.1684849995565; Tue, 23 May 2023 06:53:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849995; cv=none; d=google.com; s=arc-20160816; b=qAIptApIfidDNIhC71Oml1yixPnNDE5XOT6AZfuHk8laNk4pMEpmbcL8A8AFEhkuUC Ejde7WvxxegbucpMVhdC05FEOZ4UvZ6FY3kRdOuk6uDQ/OAcoLzpLi9XTsUHKbjPIYUf iW9FL5o2/quNJyiQ/Tr/dzDp1s6XG7jBgUC9q6/KBDIKR7IIRAvyENvs2lhjehZwE1ja 4fG/n9xhDm2oulXsWg/KNy3NH/yPupLm6q8LPsX+vHHFJpvyLRTOMizLOf+Rxt4OfKRJ eDBnSGB5TkQW95893bdKCFMey/W80PD/7X+2accAMfvK0ViKake4pvlydWgS0Q3DZ2Dk 4c8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=2DqFQEVjlcRQv07fj6FpnhLPU4q7Af597SdqlFdm/+U=; b=C1tU3dRty6/fd4pI9jP530q6nwK/r+knekQzi80ms0Lcd8ieLegxAE8I+D3fOA4B+s Mxl1XPX2JMlsKi8KaZnXjm/TYCeVfxOEtshIhZmRE0ZyipRVOhZ6dFmF8y+VdfyvipiR XhQDiXkSXPQ1Qs1WqATfr+ioCszWamMpL+huGV08ZYJI/7Op+jdtAX8Z+0KonhhTCeCr SvOTenH6eumkme3XFPy1Gs5IMDL8RTtKumJjOjSdk4bYxJfLx9/BjM4X/iWbU3FVHUK5 JGfezQtQAB8IJKAQ8MW86kY1UJQT1SK+CIG5L9vNHMN8F3KNmuCTOrRO7ecLKzltmkSv r0Hg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hcOc3ftn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e4-20020a05620a12c400b007490c58d340si4705889qkl.583.2023.05.23.06.53.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:53:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hcOc3ftn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMi-0000UP-6b; Tue, 23 May 2023 09:47:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMf-0000Hi-MS for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:49 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMd-000175-IL for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:49 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64d5f65a2f7so2014712b3a.1 for ; Tue, 23 May 2023 06:47:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849666; x=1687441666; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2DqFQEVjlcRQv07fj6FpnhLPU4q7Af597SdqlFdm/+U=; b=hcOc3ftnldIG8D8G0l6d7WeujSpQ/+j/KpzJcqYwAFWPzVBxuHn5PtdyWjf7baHmn6 CjS89sPL1mCXRu1YO2liOcWQSAa+n6xDsDCaQF0jsxvIzUMnsPmSIwhPVLFWe2z+BMoY Jy4ZTHhUNU1hl/E0XzSlsB5kbHknEkO8wrWSBbHuwgO49gVRLSOKGAP5+FTo+0lYyIDG J1RpSRMuXIQtkuCDGUSWYh7He3WF7Zzo+Ur/mXjRDWrrU+ez7jbcbYyAPrYVg6ywU+9F eDDNuqZ7K9As/kvcrc2urHMvBgdlA/o42liF/Ly36KztbsrSui6sqxq+vtj6ZmCJgWcv U/VA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849666; x=1687441666; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2DqFQEVjlcRQv07fj6FpnhLPU4q7Af597SdqlFdm/+U=; b=IVagL8MdwIQ/cCRd7u9oE0qnVqybyIV4MeffHsQODRRVFYZP/dMtSEUk7t688hEiRf oubP7jXd+q6uaX6BdhzyVjGD16TgGzXpY79P+GSrwDumL8U0op34jRVPjPRaod/h6fbV A6QGpV1j6jISe3w+7w2Qqz3sR6KsFTUOjoFVuEES9J6zR+YM50nO7IYbaJHSDGedek9Y WPdIMQpvhvbKu8olmlEUxeBVn59gMg+yXnR8ivvTGzAJ/2zKuOmnH40sqzkAo5/a30aV oZ6j5WrcwPm8X9uylqdG4+GlSWa4fdKygMgC5TRtvw6EMk5k5JkThmmiFerjLFTqjeTb VCcw== X-Gm-Message-State: AC+VfDwFNE6vDNpMUqdGazSd13L+7rHwQxHhKvXa8+xjcGff3bv0eFNo Go0eSJgoUO9hslCqSxev14jR1MKe1fSgIgKvtPA= X-Received: by 2002:a05:6a20:c701:b0:105:53:998 with SMTP id hi1-20020a056a20c70100b0010500530998mr12486354pzb.12.1684849666339; Tue, 23 May 2023 06:47:46 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 13/27] include/qemu: Move CONFIG_ATOMIC128_OPT handling to atomic128.h Date: Tue, 23 May 2023 06:47:19 -0700 Message-Id: <20230523134733.678646-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Not only the routines in ldst_atomicity.c.inc need markup, but also the ones in the headers. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/generic/host/atomic128-cas.h | 12 ++++++++---- host/include/generic/host/atomic128-ldst.h | 18 ++++++++++++------ include/qemu/atomic128.h | 17 +++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 17 ----------------- 4 files changed, 37 insertions(+), 27 deletions(-) diff --git a/host/include/generic/host/atomic128-cas.h b/host/include/generic/host/atomic128-cas.h index 513622fe34..991d3da082 100644 --- a/host/include/generic/host/atomic128-cas.h +++ b/host/include/generic/host/atomic128-cas.h @@ -12,24 +12,28 @@ #define HOST_ATOMIC128_CAS_H #if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r, c, n; c.s = cmp; n.s = new; - r.i = qatomic_cmpxchg__nocheck((__int128_t *)ptr, c.i, n.i); + r.i = qatomic_cmpxchg__nocheck(ptr_align, c.i, n.i); return r.s; } # define HAVE_CMPXCHG128 1 #elif defined(CONFIG_CMPXCHG128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r, c, n; c.s = cmp; n.s = new; - r.i = __sync_val_compare_and_swap_16((__int128_t *)ptr, c.i, n.i); + r.i = __sync_val_compare_and_swap_16(ptr_align, c.i, n.i); return r.s; } # define HAVE_CMPXCHG128 1 diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index e7354a9255..46911dfb61 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -12,32 +12,38 @@ #define HOST_ATOMIC128_LDST_H #if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_read(Int128 *ptr) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_read(Int128 *ptr) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r; - r.i = qatomic_read__nocheck((__int128_t *)ptr); + r.i = qatomic_read__nocheck(ptr_align); return r.s; } -static inline void atomic16_set(Int128 *ptr, Int128 val) +static inline void ATTRIBUTE_ATOMIC128_OPT +atomic16_set(Int128 *ptr, Int128 val) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias v; v.s = val; - qatomic_set__nocheck((__int128_t *)ptr, v.i); + qatomic_set__nocheck(ptr_align, v.i); } # define HAVE_ATOMIC128 1 #elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 -static inline Int128 atomic16_read(Int128 *ptr) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_read(Int128 *ptr) { /* Maybe replace 0 with 0, returning the old value. */ Int128 z = int128_make64(0); return atomic16_cmpxchg(ptr, z, z); } -static inline void atomic16_set(Int128 *ptr, Int128 val) +static inline void ATTRIBUTE_ATOMIC128_OPT +atomic16_set(Int128 *ptr, Int128 val) { Int128 old = *ptr, cmp; do { diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h index 3a8adb4d47..34554bf0ac 100644 --- a/include/qemu/atomic128.h +++ b/include/qemu/atomic128.h @@ -15,6 +15,23 @@ #include "qemu/int128.h" +/* + * If __alignof(unsigned __int128) < 16, GCC may refuse to inline atomics + * that are supported by the host, e.g. s390x. We can force the pointer to + * have our known alignment with __builtin_assume_aligned, however prior to + * GCC 13 that was only reliable with optimization enabled. See + * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107389 + */ +#if defined(CONFIG_ATOMIC128_OPT) +# if !defined(__OPTIMIZE__) +# define ATTRIBUTE_ATOMIC128_OPT __attribute__((optimize("O1"))) +# endif +# define CONFIG_ATOMIC128 +#endif +#ifndef ATTRIBUTE_ATOMIC128_OPT +# define ATTRIBUTE_ATOMIC128_OPT +#endif + /* * GCC is a house divided about supporting large atomic operations. * diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index ba5db7c366..b89631bbef 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -16,23 +16,6 @@ #endif #define HAVE_al8_fast (ATOMIC_REG_SIZE >= 8) -/* - * If __alignof(unsigned __int128) < 16, GCC may refuse to inline atomics - * that are supported by the host, e.g. s390x. We can force the pointer to - * have our known alignment with __builtin_assume_aligned, however prior to - * GCC 13 that was only reliable with optimization enabled. See - * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107389 - */ -#if defined(CONFIG_ATOMIC128_OPT) -# if !defined(__OPTIMIZE__) -# define ATTRIBUTE_ATOMIC128_OPT __attribute__((optimize("O1"))) -# endif -# define CONFIG_ATOMIC128 -#endif -#ifndef ATTRIBUTE_ATOMIC128_OPT -# define ATTRIBUTE_ATOMIC128_OPT -#endif - #if defined(CONFIG_ATOMIC128) # define HAVE_al16_fast true #else From patchwork Tue May 23 13:47:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684936 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730149wrt; Tue, 23 May 2023 06:51:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7rX5wnNH8bOUFagwhm336HpTU3WE0Hu2yq1w+LylWyUuZ0fKnW9JWXomQyU+ytyQrUsm+J X-Received: by 2002:ac8:7f86:0:b0:3f6:b32c:3766 with SMTP id z6-20020ac87f86000000b003f6b32c3766mr6723781qtj.11.1684849907333; Tue, 23 May 2023 06:51:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849907; cv=none; d=google.com; s=arc-20160816; b=DzoKSOkcdIGWFk+4FIpg3V6dHYdaqG6DVuVjz99o2pe9t89+/uHMiFj3Ho7quTQ/Mo n/xKMhqy3pvC8OG6KutjfHJRO50HUtMB3XtcNHKvYLb05omynyjgkthNBkLoKaX6yEV5 7OHdF/y/zIO3I0eevIftHtgE4FIsqMXl74KoMdq2l4jQaTMAuWMfoiYtxsK3LkYWhzpH OpksTlf/CIO9KxvxjjUZgs7sfGOhge8Ci/LhFTSPm9fIYCTzVhqgJ9HLt+TC3D9DMaSD 3W5DxJizv/4z/abo04fWb/hFBkLNtTOF2RzVzGu6JmkWT+3KLlPaKAW58iIFG9KRhOyy Ov1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LDPGtChIiM90g/ZXe1/z5yfO/noMZETMdebtBSg+Cq0=; b=pv3ICsHqJ4VNdLUZA4fqAEAW4FFFEIEf4liWFum5Awzk7JhKIi1cXH3MGK/30WaSu6 KgXXudr0WJr+yPzywuFKpkTa1BJtFPxpOXdBqKEUjxQqrAVyWd4Ux7hJ4gQNlO47EIu9 XQ1cPHfWISmXXsFgMr6H/OWhnlEVI9fcoDMKhpvBRK+mZTrK8BC/W/ojR/dAI4EfTvnI pmaW1jXTJZiJxbmLW/1Fd175hiC8m+yKCWq/l7vEBbFRkfQ00cRYPfRPz94TTPlGqDKf rYW7Q4igCp8KeCIXeRLvHqV+AgqKPcGuMozlMUVAKI5Zw3j5FQ6aE9fDAsKqcwZ8X6me Kcig== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Qrr8eJBv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f10-20020a05622a104a00b003e4eb9adb9dsi5453267qte.148.2023.05.23.06.51.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:51:47 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Qrr8eJBv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMh-0000U0-Ud; Tue, 23 May 2023 09:47:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMg-0000NE-3l for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:50 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMe-00013T-0z for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:49 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64d18d772bdso6812853b3a.3 for ; Tue, 23 May 2023 06:47:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849667; x=1687441667; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LDPGtChIiM90g/ZXe1/z5yfO/noMZETMdebtBSg+Cq0=; b=Qrr8eJBvPY4ej9Arezd//AWGFol956Gz9QHzsy4RejIdkbt2UnWWrqtnl7q1YpavLh rxBo+gw5xNVc5mLz05fPSdZHL3g6N4hcopbBXhb4D8e/KWfgxzWyj8sJdzakzglxHkKJ rZcJNYPOa+y4Qtm5tZCR/YGPt1VIS+fihXZC4VEApd8CG7rRtA3JLljnT6ib8VivE1EL PXtx0FLbBim9EeGor+XpiY/x/ar2XNqg71uPFrli3TBNwYr+GyaAkG1zSRCfnxdsT6gm te+kGKOAPRJLlAFNF1Cj38XFqgznzo0CCXrDEa2nBzv+vPaXD72j2S7TKdhJq6DbaEC7 Mgzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849667; x=1687441667; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LDPGtChIiM90g/ZXe1/z5yfO/noMZETMdebtBSg+Cq0=; b=UhBTSOkBRXPy1sUbWhZxj8tT81k3eRQ6NbTNrZQ8OJPs2EO+Yzf/OoOPb4uCAPLJWM l4SQDHuwgwaSyAzQEEn8BiIupV5s+XATtQn90XgljPD6lK9pKHYAU1Cu1ffbyOK3DMEq RKt1XXofc3PaVzGX9/MyO8j7nUfiH8Ma6ZZwKmU4QgRHZLsVyrGGyRLBj6UnCtCgtzqH 6ScbNIFJv4udldV40CIbb+tBUkUKd6ZuttX8LBNUtP38NV2n7jVHVMWQscEXEut+sFGs 3Kmnm28imyRTnDhX+RSLNaioF7tdjSDecLrn4uOnuL31MXn+5Cnj7umi8jZ/PLcdDCX3 wIcQ== X-Gm-Message-State: AC+VfDxzoPNUfz+d0/Fr3PI4timSIt8gkY+pgTWBzLNvHuuc9yUDNKE/ 60MR5/lTAMcTHvbkO/iaVY6K709LkHIG7GEJh6M= X-Received: by 2002:a05:6a00:a92:b0:64d:277c:77c3 with SMTP id b18-20020a056a000a9200b0064d277c77c3mr17325330pfl.23.1684849667227; Tue, 23 May 2023 06:47:47 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, qemu-ppc@nongnu.org, Daniel Henrique Barboza , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , David Gibson , Greg Kurz Subject: [PATCH v2 14/27] target/ppc: Use tcg_gen_qemu_{ld, st}_i128 for LQARX, LQ, STQ Date: Tue, 23 May 2023 06:47:20 -0700 Message-Id: <20230523134733.678646-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org No need to roll our own, as this is now provided by tcg. This was the last use of retxl, so remove that too. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- Cc: qemu-ppc@nongnu.org Cc: Daniel Henrique Barboza Cc: "Cédric Le Goater" Cc: David Gibson Cc: Greg Kurz --- target/ppc/cpu.h | 1 - target/ppc/helper.h | 9 ---- target/ppc/mem_helper.c | 48 -------------------- target/ppc/translate.c | 34 ++------------- target/ppc/translate/fixedpoint-impl.c.inc | 51 +++------------------- 5 files changed, 11 insertions(+), 132 deletions(-) diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h index 1c02596d9f..0f9f2e1a0c 100644 --- a/target/ppc/cpu.h +++ b/target/ppc/cpu.h @@ -1124,7 +1124,6 @@ struct CPUArchState { /* used to speed-up TLB assist handlers */ target_ulong nip; /* next instruction pointer */ - uint64_t retxh; /* high part of 128-bit helper return */ /* when a memory exception occurs, the access type is stored here */ int access_type; diff --git a/target/ppc/helper.h b/target/ppc/helper.h index 0beaca5c7a..38efbc351c 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -810,12 +810,3 @@ DEF_HELPER_4(DSCLIQ, void, env, fprp, fprp, i32) DEF_HELPER_1(tbegin, void, env) DEF_HELPER_FLAGS_1(fixup_thrm, TCG_CALL_NO_RWG, void, env) - -#ifdef TARGET_PPC64 -DEF_HELPER_FLAGS_3(lq_le_parallel, TCG_CALL_NO_WG, i64, env, tl, i32) -DEF_HELPER_FLAGS_3(lq_be_parallel, TCG_CALL_NO_WG, i64, env, tl, i32) -DEF_HELPER_FLAGS_5(stq_le_parallel, TCG_CALL_NO_WG, - void, env, tl, i64, i64, i32) -DEF_HELPER_FLAGS_5(stq_be_parallel, TCG_CALL_NO_WG, - void, env, tl, i64, i64, i32) -#endif diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 1578887a8f..46eae65819 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -367,54 +367,6 @@ target_ulong helper_lscbx(CPUPPCState *env, target_ulong addr, uint32_t reg, return i; } -#ifdef TARGET_PPC64 -uint64_t helper_lq_le_parallel(CPUPPCState *env, target_ulong addr, - uint32_t opidx) -{ - Int128 ret; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - ret = cpu_atomic_ldo_le_mmu(env, addr, opidx, GETPC()); - env->retxh = int128_gethi(ret); - return int128_getlo(ret); -} - -uint64_t helper_lq_be_parallel(CPUPPCState *env, target_ulong addr, - uint32_t opidx) -{ - Int128 ret; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - ret = cpu_atomic_ldo_be_mmu(env, addr, opidx, GETPC()); - env->retxh = int128_gethi(ret); - return int128_getlo(ret); -} - -void helper_stq_le_parallel(CPUPPCState *env, target_ulong addr, - uint64_t lo, uint64_t hi, uint32_t opidx) -{ - Int128 val; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - val = int128_make128(lo, hi); - cpu_atomic_sto_le_mmu(env, addr, val, opidx, GETPC()); -} - -void helper_stq_be_parallel(CPUPPCState *env, target_ulong addr, - uint64_t lo, uint64_t hi, uint32_t opidx) -{ - Int128 val; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - val = int128_make128(lo, hi); - cpu_atomic_sto_be_mmu(env, addr, val, opidx, GETPC()); -} -#endif - /*****************************************************************************/ /* Altivec extension helpers */ #if HOST_BIG_ENDIAN diff --git a/target/ppc/translate.c b/target/ppc/translate.c index f603f1a939..1720570b9b 100644 --- a/target/ppc/translate.c +++ b/target/ppc/translate.c @@ -3757,6 +3757,7 @@ static void gen_lqarx(DisasContext *ctx) { int rd = rD(ctx->opcode); TCGv EA, hi, lo; + TCGv_i128 t16; if (unlikely((rd & 1) || (rd == rA(ctx->opcode)) || (rd == rB(ctx->opcode)))) { @@ -3772,36 +3773,9 @@ static void gen_lqarx(DisasContext *ctx) lo = cpu_gpr[rd + 1]; hi = cpu_gpr[rd]; - if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { - if (HAVE_ATOMIC128) { - TCGv_i32 oi = tcg_temp_new_i32(); - if (ctx->le_mode) { - tcg_gen_movi_i32(oi, make_memop_idx(MO_LE | MO_128 | MO_ALIGN, - ctx->mem_idx)); - gen_helper_lq_le_parallel(lo, cpu_env, EA, oi); - } else { - tcg_gen_movi_i32(oi, make_memop_idx(MO_BE | MO_128 | MO_ALIGN, - ctx->mem_idx)); - gen_helper_lq_be_parallel(lo, cpu_env, EA, oi); - } - tcg_gen_ld_i64(hi, cpu_env, offsetof(CPUPPCState, retxh)); - } else { - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; - return; - } - } else if (ctx->le_mode) { - tcg_gen_qemu_ld_i64(lo, EA, ctx->mem_idx, MO_LEUQ | MO_ALIGN_16); - tcg_gen_mov_tl(cpu_reserve, EA); - gen_addr_add(ctx, EA, EA, 8); - tcg_gen_qemu_ld_i64(hi, EA, ctx->mem_idx, MO_LEUQ); - } else { - tcg_gen_qemu_ld_i64(hi, EA, ctx->mem_idx, MO_BEUQ | MO_ALIGN_16); - tcg_gen_mov_tl(cpu_reserve, EA); - gen_addr_add(ctx, EA, EA, 8); - tcg_gen_qemu_ld_i64(lo, EA, ctx->mem_idx, MO_BEUQ); - } + t16 = tcg_temp_new_i128(); + tcg_gen_qemu_ld_i128(t16, EA, ctx->mem_idx, DEF_MEMOP(MO_128 | MO_ALIGN)); + tcg_gen_extr_i128_i64(lo, hi, t16); tcg_gen_st_tl(hi, cpu_env, offsetof(CPUPPCState, reserve_val)); tcg_gen_st_tl(lo, cpu_env, offsetof(CPUPPCState, reserve_val2)); diff --git a/target/ppc/translate/fixedpoint-impl.c.inc b/target/ppc/translate/fixedpoint-impl.c.inc index 02d86b77a8..f47f1a50e8 100644 --- a/target/ppc/translate/fixedpoint-impl.c.inc +++ b/target/ppc/translate/fixedpoint-impl.c.inc @@ -72,7 +72,7 @@ static bool do_ldst_quad(DisasContext *ctx, arg_D *a, bool store, bool prefixed) #if defined(TARGET_PPC64) TCGv ea; TCGv_i64 low_addr_gpr, high_addr_gpr; - MemOp mop; + TCGv_i128 t16; REQUIRE_INSNS_FLAGS(ctx, 64BX); @@ -101,51 +101,14 @@ static bool do_ldst_quad(DisasContext *ctx, arg_D *a, bool store, bool prefixed) low_addr_gpr = cpu_gpr[a->rt + 1]; high_addr_gpr = cpu_gpr[a->rt]; } + t16 = tcg_temp_new_i128(); - if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { - if (HAVE_ATOMIC128) { - mop = DEF_MEMOP(MO_128); - TCGv_i32 oi = tcg_constant_i32(make_memop_idx(mop, ctx->mem_idx)); - if (store) { - if (ctx->le_mode) { - gen_helper_stq_le_parallel(cpu_env, ea, low_addr_gpr, - high_addr_gpr, oi); - } else { - gen_helper_stq_be_parallel(cpu_env, ea, high_addr_gpr, - low_addr_gpr, oi); - - } - } else { - if (ctx->le_mode) { - gen_helper_lq_le_parallel(low_addr_gpr, cpu_env, ea, oi); - tcg_gen_ld_i64(high_addr_gpr, cpu_env, - offsetof(CPUPPCState, retxh)); - } else { - gen_helper_lq_be_parallel(high_addr_gpr, cpu_env, ea, oi); - tcg_gen_ld_i64(low_addr_gpr, cpu_env, - offsetof(CPUPPCState, retxh)); - } - } - } else { - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; - } + if (store) { + tcg_gen_concat_i64_i128(t16, low_addr_gpr, high_addr_gpr); + tcg_gen_qemu_st_i128(t16, ea, ctx->mem_idx, DEF_MEMOP(MO_128)); } else { - mop = DEF_MEMOP(MO_UQ); - if (store) { - tcg_gen_qemu_st_i64(low_addr_gpr, ea, ctx->mem_idx, mop); - } else { - tcg_gen_qemu_ld_i64(low_addr_gpr, ea, ctx->mem_idx, mop); - } - - gen_addr_add(ctx, ea, ea, 8); - - if (store) { - tcg_gen_qemu_st_i64(high_addr_gpr, ea, ctx->mem_idx, mop); - } else { - tcg_gen_qemu_ld_i64(high_addr_gpr, ea, ctx->mem_idx, mop); - } + tcg_gen_qemu_ld_i128(t16, ea, ctx->mem_idx, DEF_MEMOP(MO_128)); + tcg_gen_extr_i128_i64(low_addr_gpr, high_addr_gpr, t16); } #else qemu_build_not_reached(); From patchwork Tue May 23 13:47:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684924 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728673wrt; Tue, 23 May 2023 06:48:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5rpsHVUxzxwu4NycxFLI/s3UEFeqwY1Aru6/qswxPBXbPGGzV8odsXridFXao3dhpb7h1C X-Received: by 2002:a05:622a:40e:b0:3f6:913e:5535 with SMTP id n14-20020a05622a040e00b003f6913e5535mr23005415qtx.62.1684849708971; Tue, 23 May 2023 06:48:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849708; cv=none; d=google.com; s=arc-20160816; b=jy8VNoeyzsznGYXfP9jTwAX2wdmLlzSm+vq6FJIoX/hm+Wg7SYmMZfC/BfZgNaW1Ay /JB8M/DQy3dFHmlYHD7qctGkMv7o6xvxh9G8l7Ra9bWRN6d3DVtPQ2uiwzqik7qjP4rf CIQUGYg+2SOJSNw6kjeHNja1UIb7G4LT1t04MUhajIZgvuqNweoFihVXHHCEk03pa1bo XOcSk0NX8d6RoUe1wNRuxkVV3Ew1lHUputBbp1Ht8dzISYpa5hNyqreMBz/6ezBwQHeY BEjj4/jnlyrjXbBWy15lf3bTgzBQeqcPXSb9hzlfzGbox3CTfp4vDBN4Bwo2yQnTl3Nu gAMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=xmt+ITqZqh7I7gFJaXBYqdm/Oa0NR4029Dhdaw3mjBo=; b=O2KTUH20bXyf69w0/dGalCr0gD1KlCpKqebnYnXi2EEWGhaOrV3rtDsA9TR0RcBqH/ mK4U8sb97MRpuqn8UqPdHX6z1JaaPPVAp+o3Nr1+z1IIf00H98Phu0nhe44Gmf+Ibc18 6uv9/WwDlp7Am5hNqbveOyAeKfmiQ2CRiAEULix9V6LJ8B9wFJmaUiK85SF8Cz1N8ttz FkzK333h7/tjzcxMRLM8G6qlZtYn1gj/hn7Wb6nUPO8zuFi6bKEOw56qinSLzu0v2Rzs eSRjTvpsMFfxQNG8fxjVr656c3mWrTiYvvBl9A5tEVA5MFOBJVH35IZU4veNls/cQrd4 ZJhw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="yOV1W/Wf"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j17-20020ac85f91000000b003f504ae15d3si5044626qta.262.2023.05.23.06.48.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="yOV1W/Wf"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMj-0000WA-2k; Tue, 23 May 2023 09:47:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMh-0000S9-CB for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:51 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMe-00016H-TS for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:51 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64d5b4c400fso3386055b3a.1 for ; Tue, 23 May 2023 06:47:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849668; x=1687441668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xmt+ITqZqh7I7gFJaXBYqdm/Oa0NR4029Dhdaw3mjBo=; b=yOV1W/Wfqk0niZ8IHPJB4v08TDZBCIwZ2h6rAyTqzzKoiyZL4IeAoVzJlRyTebCYe6 H+4zh/Od9ZOFG5BmYItgXJBa+SEoKwr9pWpa78KmAYug1xItheB1KUdeWYho4ZdMIWvB zB4uEEwCF6uWQCl9U6BdeaOHa8Szbg7Q3FuzwGJXC2mAaVqHpUJQDaYHhMnx5LkODzIq GERxz5W6pa87vEUHITTMLiWfp+hiu42Wr7p6E97X1MyfH+lNZrhhuLDy2O2mmOB+W9wi VO4AdP3whvopBRKTqh7UQ335o6d7g7E+Qv3AleuWl4hsFtz1dMWGCHZnbh0gUWi65VbX 50Vw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849668; x=1687441668; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xmt+ITqZqh7I7gFJaXBYqdm/Oa0NR4029Dhdaw3mjBo=; b=M0Qort4aQZECl8mQBfjOYhj/gsW3dZPKV2Zi9auQ/LCoxnq+DPYMnhEjMWybpwCzSp +GIhOHbKB1Tfvgmh1lj2IiavQ+wQnxHqnk9rAxN3gifpGh6mmJeWKT47wFdjhd7bcAoP oFd3cQ0Qa9QMWEGH4Yvjhn842lWfEyN5eLWwL98Z90cgdLVw6ILDDZTaEGFvpFSW1pQn a77UwDgVEUiP+XfnYDqcPj6ozLq1umusPrcnyFdKhAYODCarPyALt5wfnZPzrKI6rkqd tunO8gX6XNyOglZ8kSH3wyGN3ZpM/7FX403MNiu6D7dwb7cf/yUVseP3GYu9SOUzHS9a fyYQ== X-Gm-Message-State: AC+VfDyv5ZiwK4qOmGnLzyKTppjCQ39qhoLzfkuj3jv8bfO13qH4qDOe pwuKGzTamUhf40AGhZICb2ubBvxpfW50BZieeNc= X-Received: by 2002:a05:6a21:788f:b0:10b:7400:cef7 with SMTP id bf15-20020a056a21788f00b0010b7400cef7mr7563275pzc.17.1684849668115; Tue, 23 May 2023 06:47:48 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, qemu-s390x@nongnu.org, David Hildenbrand , Ilya Leoshkevich Subject: [PATCH v2 15/27] target/s390x: Use tcg_gen_qemu_{ld, st}_i128 for LPQ, STPQ Date: Tue, 23 May 2023 06:47:21 -0700 Message-Id: <20230523134733.678646-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org No need to roll our own, as this is now provided by tcg. This was the last use of retxl, so remove that too. Signed-off-by: Richard Henderson Reviewed-by: David Hildenbrand Reviewed-by: Alex Bennée --- Cc: qemu-s390x@nongnu.org Cc: David Hildenbrand Cc: Ilya Leoshkevich --- target/s390x/cpu.h | 3 -- target/s390x/helper.h | 4 --- target/s390x/tcg/mem_helper.c | 61 -------------------------------- target/s390x/tcg/translate.c | 30 +++++----------- target/s390x/tcg/insn-data.h.inc | 2 +- 5 files changed, 9 insertions(+), 91 deletions(-) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index c47e7adcb1..f130c29f83 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -76,9 +76,6 @@ struct CPUArchState { float_status fpu_status; /* passed to softfloat lib */ - /* The low part of a 128-bit return, or remainder of a divide. */ - uint64_t retxl; - PSW psw; S390CrashReason crash_reason; diff --git a/target/s390x/helper.h b/target/s390x/helper.h index 341bc51ec2..7529e725f2 100644 --- a/target/s390x/helper.h +++ b/target/s390x/helper.h @@ -108,10 +108,6 @@ DEF_HELPER_FLAGS_2(sfas, TCG_CALL_NO_WG, void, env, i64) DEF_HELPER_FLAGS_2(srnm, TCG_CALL_NO_WG, void, env, i64) DEF_HELPER_FLAGS_1(popcnt, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_2(stfle, i32, env, i64) -DEF_HELPER_FLAGS_2(lpq, TCG_CALL_NO_WG, i64, env, i64) -DEF_HELPER_FLAGS_2(lpq_parallel, TCG_CALL_NO_WG, i64, env, i64) -DEF_HELPER_FLAGS_4(stpq, TCG_CALL_NO_WG, void, env, i64, i64, i64) -DEF_HELPER_FLAGS_4(stpq_parallel, TCG_CALL_NO_WG, void, env, i64, i64, i64) DEF_HELPER_4(mvcos, i32, env, i64, i64, i64) DEF_HELPER_4(cu12, i32, env, i32, i32, i32) DEF_HELPER_4(cu14, i32, env, i32, i32, i32) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 8b58b8d88d..0e0d66b3b6 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -2398,67 +2398,6 @@ uint64_t HELPER(lra)(CPUS390XState *env, uint64_t addr) } #endif -/* load pair from quadword */ -uint64_t HELPER(lpq)(CPUS390XState *env, uint64_t addr) -{ - uintptr_t ra = GETPC(); - uint64_t hi, lo; - - check_alignment(env, addr, 16, ra); - hi = cpu_ldq_data_ra(env, addr + 0, ra); - lo = cpu_ldq_data_ra(env, addr + 8, ra); - - env->retxl = lo; - return hi; -} - -uint64_t HELPER(lpq_parallel)(CPUS390XState *env, uint64_t addr) -{ - uintptr_t ra = GETPC(); - uint64_t hi, lo; - int mem_idx; - MemOpIdx oi; - Int128 v; - - assert(HAVE_ATOMIC128); - - mem_idx = cpu_mmu_index(env, false); - oi = make_memop_idx(MO_TEUQ | MO_ALIGN_16, mem_idx); - v = cpu_atomic_ldo_be_mmu(env, addr, oi, ra); - hi = int128_gethi(v); - lo = int128_getlo(v); - - env->retxl = lo; - return hi; -} - -/* store pair to quadword */ -void HELPER(stpq)(CPUS390XState *env, uint64_t addr, - uint64_t low, uint64_t high) -{ - uintptr_t ra = GETPC(); - - check_alignment(env, addr, 16, ra); - cpu_stq_data_ra(env, addr + 0, high, ra); - cpu_stq_data_ra(env, addr + 8, low, ra); -} - -void HELPER(stpq_parallel)(CPUS390XState *env, uint64_t addr, - uint64_t low, uint64_t high) -{ - uintptr_t ra = GETPC(); - int mem_idx; - MemOpIdx oi; - Int128 v; - - assert(HAVE_ATOMIC128); - - mem_idx = cpu_mmu_index(env, false); - oi = make_memop_idx(MO_TEUQ | MO_ALIGN_16, mem_idx); - v = int128_make128(low, high); - cpu_atomic_sto_be_mmu(env, addr, v, oi, ra); -} - /* Execute instruction. This instruction executes an insn modified with the contents of r1. It does not change the executed instruction in memory; it does not change the program counter. diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c index d6670e6a87..3eb3708d55 100644 --- a/target/s390x/tcg/translate.c +++ b/target/s390x/tcg/translate.c @@ -335,11 +335,6 @@ static void store_freg32_i64(int reg, TCGv_i64 v) tcg_gen_st32_i64(v, cpu_env, freg32_offset(reg)); } -static void return_low128(TCGv_i64 dest) -{ - tcg_gen_ld_i64(dest, cpu_env, offsetof(CPUS390XState, retxl)); -} - static void update_psw_addr(DisasContext *s) { /* psw.addr */ @@ -3130,15 +3125,9 @@ static DisasJumpType op_lpd(DisasContext *s, DisasOps *o) static DisasJumpType op_lpq(DisasContext *s, DisasOps *o) { - if (!(tb_cflags(s->base.tb) & CF_PARALLEL)) { - gen_helper_lpq(o->out, cpu_env, o->in2); - } else if (HAVE_ATOMIC128) { - gen_helper_lpq_parallel(o->out, cpu_env, o->in2); - } else { - gen_helper_exit_atomic(cpu_env); - return DISAS_NORETURN; - } - return_low128(o->out2); + o->out_128 = tcg_temp_new_i128(); + tcg_gen_qemu_ld_i128(o->out_128, o->in2, get_mem_index(s), + MO_TE | MO_128 | MO_ALIGN); return DISAS_NEXT; } @@ -4533,14 +4522,11 @@ static DisasJumpType op_stmh(DisasContext *s, DisasOps *o) static DisasJumpType op_stpq(DisasContext *s, DisasOps *o) { - if (!(tb_cflags(s->base.tb) & CF_PARALLEL)) { - gen_helper_stpq(cpu_env, o->in2, o->out2, o->out); - } else if (HAVE_ATOMIC128) { - gen_helper_stpq_parallel(cpu_env, o->in2, o->out2, o->out); - } else { - gen_helper_exit_atomic(cpu_env); - return DISAS_NORETURN; - } + TCGv_i128 t16 = tcg_temp_new_i128(); + + tcg_gen_concat_i64_i128(t16, o->out2, o->out); + tcg_gen_qemu_st_i128(t16, o->in2, get_mem_index(s), + MO_TE | MO_128 | MO_ALIGN); return DISAS_NEXT; } diff --git a/target/s390x/tcg/insn-data.h.inc b/target/s390x/tcg/insn-data.h.inc index 1f1ac742a9..bcc70d99ba 100644 --- a/target/s390x/tcg/insn-data.h.inc +++ b/target/s390x/tcg/insn-data.h.inc @@ -570,7 +570,7 @@ D(0xc804, LPD, SSF, ILA, 0, 0, new_P, r3_P32, lpd, 0, MO_TEUL) D(0xc805, LPDG, SSF, ILA, 0, 0, new_P, r3_P64, lpd, 0, MO_TEUQ) /* LOAD PAIR FROM QUADWORD */ - C(0xe38f, LPQ, RXY_a, Z, 0, a2, r1_P, 0, lpq, 0) + C(0xe38f, LPQ, RXY_a, Z, 0, a2, 0, r1_D64, lpq, 0) /* LOAD POSITIVE */ C(0x1000, LPR, RR_a, Z, 0, r2_32s, new, r1_32, abs, abs32) C(0xb900, LPGR, RRE, Z, 0, r2, r1, 0, abs, abs64) From patchwork Tue May 23 13:47:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684932 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729821wrt; Tue, 23 May 2023 06:50:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4Sa9Jn8ACDCaL8LCzKT41rydYkm/r62sEl65g2He/B1XeIlCOQZ7+J7Pqhw9JqlMcJ7Pc6 X-Received: by 2002:ad4:594c:0:b0:5ef:8161:cb9a with SMTP id eo12-20020ad4594c000000b005ef8161cb9amr28130207qvb.7.1684849858802; Tue, 23 May 2023 06:50:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849858; cv=none; d=google.com; s=arc-20160816; b=LsxtODv6iHh2Y83Qfl491xsN2hVyAot4+OTXZmbsAaIdQHGyAB84IHNm3/LdOw7xK2 snIdmATkmkFbqFkUkd8GqLFneVk/NOC7Z4NXAPUcw7ZXuhHHCP5Q3J6gyPq70mzNc3h8 eHcvks/EwhKqMU38QjeUP0aVICCe/45gIaXIgA+zdXBOPJDkp1IecqJhSDo4lKqV6MlA blFvvnttmQLUI9Jh9FtWb7iReyACrBYayoT9aZGihZr9Bg3T9e1ZlvfGg3vS6lLxoMJa ypcpxrjm+0SYU/6rd5fOY7bC5gCur9qx43kZd9ctnLc4Z+r/MFFBl6m1fWPtHX7uUAML nVsA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=NuFgFDxh+RzZsMut/UZE5wwTmeNr5ObnWQi6N44NQjM=; b=HhMSHgwiuaxOV1D3ew6XL9nQcfipCl/+w6l+l9VobZmM1VCmKummHre1YgirFp/sPi c/ktsRONTNlx9tdXmXpc40MZs6BhtzbbbHvlgrFQLmUGwa65iYCmtTMfwyxFVx7RG72O Sv0g7KKco5Gpe6eCBEaYiC16sDAagAMTPwI8OWyj2P2ub1Ysf+5ao9EZT0ppqIvQYcrY 3lXIzfcTGbhJVRhXhkmz9Y4U9wuXLScCLgeYV8dXIftqrADn3nqrqei/mIAHGNvZnFVL lIl7zsy9wGzi/mGI/Po7mBLKiZlTZx1/DUV1rpK+Br0IewQ78Ch09BSeLcqHxkwrNXhs zsVA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uT52eZGq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x15-20020ae9f80f000000b007591c679e40si4774199qkh.358.2023.05.23.06.50.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:50:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uT52eZGq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMm-0000bw-0e; Tue, 23 May 2023 09:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMj-0000Xd-PA for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:53 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMg-00013F-3E for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:53 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d24136685so3499061b3a.1 for ; Tue, 23 May 2023 06:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849669; x=1687441669; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=NuFgFDxh+RzZsMut/UZE5wwTmeNr5ObnWQi6N44NQjM=; b=uT52eZGqLaXmNHaaM04D55TuXEeVBM4oBja3QGD16T1AcUJN7ZF5pgvP+JPiBXnVTb cfOKsNboi98n3Pc54fXIztP3fptovjL5F7oYr7fJEbrrKDcoaPLYHDghvg0TWoI2YxKe +1lbI3EEurTbFrDq9RotJ7qVkctokyVeDaiYr9OZlwNeHSadtxuCTgC7O/gPXYvSnsF/ SSQnGrbzKeURyCQgRh0IIAf8QxdeyRkCFy8DEo5G0X0X3+VzECSqP3WLOMuujvAVCp0B bLzcewO/zkvEQGkiguTJcGBytIEt2d+BHBd3ZMEmrJdTxm51CLSTdNapbOT2DtDBLaIu anwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849669; x=1687441669; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NuFgFDxh+RzZsMut/UZE5wwTmeNr5ObnWQi6N44NQjM=; b=BFEViseqd0jj5jkXEyeF3VRD3JuvU1xTpX3n3D/dcMPq46YxqA4JifQqHB+H/CUj5D 0MzvPhKrH3o90RDuWaD1oM12IgiGx2iKTynJ0TMaYwBGL6mAm0snDSwv8lg+hkar5n4f ZUCONN8Litje/GL0bsNrOR1klu4E4MkvWoYKCPhXExD9vdtQ2Mef9N0WRx8bUiDaO+P+ t4QjyiaVk+smuIUPl7fzulFykpROTRTImPLyNNP9if4eLdAnroT9q5QAPNICcxhLZ+sY 30gSgi8md+KjF1jVd3VwodtFtQv6OZtAdiayVz5k7LnPlOoxoh1phgXbdjXBIoCQur6i X/Qw== X-Gm-Message-State: AC+VfDzwiLVBXLAnoUahiVap/Mn9/O4IlRRa9pkeLRMaRI93RxKbXKnY NUTYfxH3tWAyy0if6d/Zy6PQHjYcCr0zO4sdZAw= X-Received: by 2002:a05:6a00:1594:b0:646:6e40:b421 with SMTP id u20-20020a056a00159400b006466e40b421mr15340032pfk.1.1684849669071; Tue, 23 May 2023 06:47:49 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v2 16/27] accel/tcg: Unify cpu_{ld,st}*_{be,le}_mmu Date: Tue, 23 May 2023 06:47:22 -0700 Message-Id: <20230523134733.678646-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With the current structure of cputlb.c, there is no difference between the little-endian and big-endian entry points, aside from the assert. Unify the pairs of functions. The only use of the functions with explicit endianness was in target/sparc64, and that was only to satisfy the assert: the correct endianness is already built into memop. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- include/exec/cpu_ldst.h | 58 ++----- accel/tcg/cputlb.c | 122 +++----------- accel/tcg/user-exec.c | 322 ++++++++++-------------------------- target/arm/tcg/m_helper.c | 4 +- target/sparc/ldst_helper.c | 18 +- accel/tcg/ldst_common.c.inc | 24 +-- 6 files changed, 137 insertions(+), 411 deletions(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 7c867c94c3..fc1d3d9301 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -207,43 +207,21 @@ void cpu_stq_le_mmuidx_ra(CPUArchState *env, abi_ptr ptr, uint64_t val, int mmu_idx, uintptr_t ra); uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); -uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); - -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra); -Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra); +uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); +uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); +uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); +Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra); void cpu_stb_mmu(CPUArchState *env, abi_ptr ptr, uint8_t val, MemOpIdx oi, uintptr_t ra); -void cpu_stw_be_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stl_be_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stq_be_mmu(CPUArchState *env, abi_ptr ptr, uint64_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stw_le_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stl_le_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stq_le_mmu(CPUArchState *env, abi_ptr ptr, uint64_t val, - MemOpIdx oi, uintptr_t ra); - -void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOpIdx oi, uintptr_t ra); -void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOpIdx oi, uintptr_t ra); +void cpu_stw_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val, + MemOpIdx oi, uintptr_t ra); +void cpu_stl_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val, + MemOpIdx oi, uintptr_t ra); +void cpu_stq_mmu(CPUArchState *env, abi_ptr ptr, uint64_t val, + MemOpIdx oi, uintptr_t ra); +void cpu_st16_mmu(CPUArchState *env, abi_ptr addr, Int128 val, + MemOpIdx oi, uintptr_t ra); uint32_t cpu_atomic_cmpxchgb_mmu(CPUArchState *env, target_ulong addr, uint32_t cmpv, uint32_t newv, @@ -416,9 +394,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_ldsw_mmuidx_ra cpu_ldsw_be_mmuidx_ra # define cpu_ldl_mmuidx_ra cpu_ldl_be_mmuidx_ra # define cpu_ldq_mmuidx_ra cpu_ldq_be_mmuidx_ra -# define cpu_ldw_mmu cpu_ldw_be_mmu -# define cpu_ldl_mmu cpu_ldl_be_mmu -# define cpu_ldq_mmu cpu_ldq_be_mmu # define cpu_stw_data cpu_stw_be_data # define cpu_stl_data cpu_stl_be_data # define cpu_stq_data cpu_stq_be_data @@ -428,9 +403,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_stw_mmuidx_ra cpu_stw_be_mmuidx_ra # define cpu_stl_mmuidx_ra cpu_stl_be_mmuidx_ra # define cpu_stq_mmuidx_ra cpu_stq_be_mmuidx_ra -# define cpu_stw_mmu cpu_stw_be_mmu -# define cpu_stl_mmu cpu_stl_be_mmu -# define cpu_stq_mmu cpu_stq_be_mmu #else # define cpu_lduw_data cpu_lduw_le_data # define cpu_ldsw_data cpu_ldsw_le_data @@ -444,9 +416,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_ldsw_mmuidx_ra cpu_ldsw_le_mmuidx_ra # define cpu_ldl_mmuidx_ra cpu_ldl_le_mmuidx_ra # define cpu_ldq_mmuidx_ra cpu_ldq_le_mmuidx_ra -# define cpu_ldw_mmu cpu_ldw_le_mmu -# define cpu_ldl_mmu cpu_ldl_le_mmu -# define cpu_ldq_mmu cpu_ldq_le_mmu # define cpu_stw_data cpu_stw_le_data # define cpu_stl_data cpu_stl_le_data # define cpu_stq_data cpu_stq_le_data @@ -456,9 +425,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_stw_mmuidx_ra cpu_stw_le_mmuidx_ra # define cpu_stl_mmuidx_ra cpu_stl_le_mmuidx_ra # define cpu_stq_mmuidx_ra cpu_stq_le_mmuidx_ra -# define cpu_stw_mmu cpu_stw_le_mmu -# define cpu_stl_mmu cpu_stl_le_mmu -# define cpu_stq_mmu cpu_stq_le_mmu #endif uint8_t cpu_ldb_code_mmu(CPUArchState *env, abi_ptr addr, diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ae0fbcdee2..b1e13d165c 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2575,89 +2575,45 @@ uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) return ret; } -uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { uint16_t ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUW); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); ret = do_ld2_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; } -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { uint32_t ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUL); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); ret = do_ld4_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; } -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { uint64_t ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUQ); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); ret = do_ld8_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; } -uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - uint16_t ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUW); - ret = do_ld2_mmu(env, addr, oi, ra, MMU_DATA_LOAD); - plugin_load_cb(env, addr, oi); - return ret; -} - -uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - uint32_t ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUL); - ret = do_ld4_mmu(env, addr, oi, ra, MMU_DATA_LOAD); - plugin_load_cb(env, addr, oi); - return ret; -} - -uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - uint64_t ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUQ); - ret = do_ld8_mmu(env, addr, oi, ra, MMU_DATA_LOAD); - plugin_load_cb(env, addr, oi); - return ret; -} - -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { Int128 ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_BE|MO_128)); - ret = do_ld16_mmu(env, addr, oi, ra); - plugin_load_cb(env, addr, oi); - return ret; -} - -Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - Int128 ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_LE|MO_128)); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); ret = do_ld16_mmu(env, addr, oi, ra); plugin_load_cb(env, addr, oi); return ret; @@ -3045,66 +3001,34 @@ void cpu_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, plugin_store_cb(env, addr, oi); } -void cpu_stw_be_mmu(CPUArchState *env, target_ulong addr, uint16_t val, - MemOpIdx oi, uintptr_t retaddr) +void cpu_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, + MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUW); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); do_st2_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } -void cpu_stl_be_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void cpu_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUL); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); do_st4_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } -void cpu_stq_be_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr) +void cpu_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, + MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUQ); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); do_st8_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } -void cpu_stw_le_mmu(CPUArchState *env, target_ulong addr, uint16_t val, - MemOpIdx oi, uintptr_t retaddr) +void cpu_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUW); - do_st2_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_stl_le_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUL); - do_st4_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_stq_le_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUQ); - do_st8_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_st16_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_BE|MO_128)); - do_st16_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_st16_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_LE|MO_128)); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); do_st16_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 36ad8284a5..19c2849c21 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -940,8 +940,8 @@ uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, return ret; } -static uint16_t do_ld2_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static uint16_t do_ld2_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint16_t ret; @@ -950,59 +950,35 @@ static uint16_t do_ld2_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_2(env, ra, haddr, mop); clear_helper_retaddr(); + + if (mop & MO_BSWAP) { + ret = bswap16(ret); + } return ret; } tcg_target_ulong helper_lduw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint16_t ret = do_ld2_he_mmu(env, addr, mop, ra); - - if (mop & MO_BSWAP) { - ret = bswap16(ret); - } - return ret; + return do_ld2_mmu(env, addr, get_memop(oi), ra); } tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - int16_t ret = do_ld2_he_mmu(env, addr, mop, ra); + return (int16_t)do_ld2_mmu(env, addr, get_memop(oi), ra); +} - if (mop & MO_BSWAP) { - ret = bswap16(ret); - } +uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + uint16_t ret = do_ld2_mmu(env, addr, get_memop(oi), ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return ret; } -uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint16_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld2_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be16(ret); -} - -uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint16_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld2_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_le16(ret); -} - -static uint32_t do_ld4_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static uint32_t do_ld4_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint32_t ret; @@ -1011,59 +987,35 @@ static uint32_t do_ld4_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_4(env, ra, haddr, mop); clear_helper_retaddr(); + + if (mop & MO_BSWAP) { + ret = bswap32(ret); + } return ret; } tcg_target_ulong helper_ldul_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint32_t ret = do_ld4_he_mmu(env, addr, mop, ra); - - if (mop & MO_BSWAP) { - ret = bswap32(ret); - } - return ret; + return do_ld4_mmu(env, addr, get_memop(oi), ra); } tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - int32_t ret = do_ld4_he_mmu(env, addr, mop, ra); + return (int32_t)do_ld4_mmu(env, addr, get_memop(oi), ra); +} - if (mop & MO_BSWAP) { - ret = bswap32(ret); - } +uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + uint32_t ret = do_ld4_mmu(env, addr, get_memop(oi), ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return ret; } -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint32_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld4_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be32(ret); -} - -uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint32_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld4_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_le32(ret); -} - -static uint64_t do_ld8_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static uint64_t do_ld8_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint64_t ret; @@ -1072,14 +1024,6 @@ static uint64_t do_ld8_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_8(env, ra, haddr, mop); clear_helper_retaddr(); - return ret; -} - -uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint64_t ret = do_ld8_he_mmu(env, addr, mop, ra); if (mop & MO_BSWAP) { ret = bswap64(ret); @@ -1087,32 +1031,22 @@ uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, return ret; } -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, +uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint64_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld8_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be64(ret); + return do_ld8_mmu(env, addr, get_memop(oi), ra); } -uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint64_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld8_he_mmu(env, addr, mop, ra); + uint64_t ret = do_ld8_mmu(env, addr, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_le64(ret); + return ret; } -static Int128 do_ld16_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static Int128 do_ld16_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; Int128 ret; @@ -1121,14 +1055,6 @@ static Int128 do_ld16_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_16(env, ra, haddr, mop); clear_helper_retaddr(); - return ret; -} - -Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - Int128 ret = do_ld16_he_mmu(env, addr, mop, ra); if (mop & MO_BSWAP) { ret = bswap128(ret); @@ -1136,38 +1062,22 @@ Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, return ret; } +Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, + MemOpIdx oi, uintptr_t ra) +{ + return do_ld16_mmu(env, addr, get_memop(oi), ra); +} + Int128 helper_ld_i128(CPUArchState *env, uint64_t addr, MemOpIdx oi) { return helper_ld16_mmu(env, addr, oi, GETPC()); } -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - Int128 ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld16_he_mmu(env, addr, mop, ra); + Int128 ret = do_ld16_mmu(env, addr, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - if (!HOST_BIG_ENDIAN) { - ret = bswap128(ret); - } - return ret; -} - -Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - Int128 ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld16_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - if (HOST_BIG_ENDIAN) { - ret = bswap128(ret); - } return ret; } @@ -1195,13 +1105,17 @@ void cpu_stb_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -static void do_st2_he_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, - MemOp mop, uintptr_t ra) +static void do_st2_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_16); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap16(val); + } store_atom_2(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1209,41 +1123,27 @@ static void do_st2_he_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, void helper_stw_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap16(val); - } - do_st2_he_mmu(env, addr, val, mop, ra); + do_st2_mmu(env, addr, val, get_memop(oi), ra); } -void cpu_stw_be_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, +void cpu_stw_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - do_st2_he_mmu(env, addr, be16_to_cpu(val), mop, ra); + do_st2_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_stw_le_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - do_st2_he_mmu(env, addr, le16_to_cpu(val), mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void do_st4_he_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOp mop, uintptr_t ra) +static void do_st4_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_32); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap32(val); + } store_atom_4(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1251,41 +1151,27 @@ static void do_st4_he_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, void helper_stl_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap32(val); - } - do_st4_he_mmu(env, addr, val, mop, ra); + do_st4_mmu(env, addr, val, get_memop(oi), ra); } -void cpu_stl_be_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOpIdx oi, uintptr_t ra) +void cpu_stl_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - do_st4_he_mmu(env, addr, be32_to_cpu(val), mop, ra); + do_st4_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_stl_le_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - do_st4_he_mmu(env, addr, le32_to_cpu(val), mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void do_st8_he_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, - MemOp mop, uintptr_t ra) +static void do_st8_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_64); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap64(val); + } store_atom_8(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1293,41 +1179,27 @@ static void do_st8_he_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, void helper_stq_mmu(CPUArchState *env, uint64_t addr, uint64_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap64(val); - } - do_st8_he_mmu(env, addr, val, mop, ra); + do_st8_mmu(env, addr, val, get_memop(oi), ra); } -void cpu_stq_be_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, +void cpu_stq_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - do_st8_he_mmu(env, addr, cpu_to_be64(val), mop, ra); + do_st8_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_stq_le_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - do_st8_he_mmu(env, addr, cpu_to_le64(val), mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void do_st16_he_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOp mop, uintptr_t ra) +static void do_st16_mmu(CPUArchState *env, abi_ptr addr, Int128 val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_128); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap128(val); + } store_atom_16(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1335,12 +1207,7 @@ static void do_st16_he_mmu(CPUArchState *env, abi_ptr addr, Int128 val, void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap128(val); - } - do_st16_he_mmu(env, addr, val, mop, ra); + do_st16_mmu(env, addr, val, get_memop(oi), ra); } void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi) @@ -1348,29 +1215,10 @@ void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi) helper_st16_mmu(env, addr, val, oi, GETPC()); } -void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, - Int128 val, MemOpIdx oi, uintptr_t ra) +void cpu_st16_mmu(CPUArchState *env, abi_ptr addr, + Int128 val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - if (!HOST_BIG_ENDIAN) { - val = bswap128(val); - } - do_st16_he_mmu(env, addr, val, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, - Int128 val, MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - if (HOST_BIG_ENDIAN) { - val = bswap128(val); - } - do_st16_he_mmu(env, addr, val, mop, ra); + do_st16_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } diff --git a/target/arm/tcg/m_helper.c b/target/arm/tcg/m_helper.c index 9758f225d6..9cef70e5c9 100644 --- a/target/arm/tcg/m_helper.c +++ b/target/arm/tcg/m_helper.c @@ -1937,8 +1937,8 @@ static bool do_v7m_function_return(ARMCPU *cpu) */ mmu_idx = arm_v7m_mmu_idx_for_secstate(env, true); oi = make_memop_idx(MO_LEUL, arm_to_core_mmu_idx(mmu_idx)); - newpc = cpu_ldl_le_mmu(env, frameptr, oi, 0); - newpsr = cpu_ldl_le_mmu(env, frameptr + 4, oi, 0); + newpc = cpu_ldl_mmu(env, frameptr, oi, 0); + newpsr = cpu_ldl_mmu(env, frameptr + 4, oi, 0); /* Consistency checks on new IPSR */ newpsr_exc = newpsr & XPSR_EXCP; diff --git a/target/sparc/ldst_helper.c b/target/sparc/ldst_helper.c index 7972d56a72..981a47d8bb 100644 --- a/target/sparc/ldst_helper.c +++ b/target/sparc/ldst_helper.c @@ -1334,25 +1334,13 @@ uint64_t helper_ld_asi(CPUSPARCState *env, target_ulong addr, ret = cpu_ldb_mmu(env, addr, oi, GETPC()); break; case 2: - if (asi & 8) { - ret = cpu_ldw_le_mmu(env, addr, oi, GETPC()); - } else { - ret = cpu_ldw_be_mmu(env, addr, oi, GETPC()); - } + ret = cpu_ldw_mmu(env, addr, oi, GETPC()); break; case 4: - if (asi & 8) { - ret = cpu_ldl_le_mmu(env, addr, oi, GETPC()); - } else { - ret = cpu_ldl_be_mmu(env, addr, oi, GETPC()); - } + ret = cpu_ldl_mmu(env, addr, oi, GETPC()); break; case 8: - if (asi & 8) { - ret = cpu_ldq_le_mmu(env, addr, oi, GETPC()); - } else { - ret = cpu_ldq_be_mmu(env, addr, oi, GETPC()); - } + ret = cpu_ldq_mmu(env, addr, oi, GETPC()); break; default: g_assert_not_reached(); diff --git a/accel/tcg/ldst_common.c.inc b/accel/tcg/ldst_common.c.inc index 6ac8d871a3..5f8144b33a 100644 --- a/accel/tcg/ldst_common.c.inc +++ b/accel/tcg/ldst_common.c.inc @@ -26,7 +26,7 @@ uint32_t cpu_lduw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUW | MO_UNALN, mmu_idx); - return cpu_ldw_be_mmu(env, addr, oi, ra); + return cpu_ldw_mmu(env, addr, oi, ra); } int cpu_ldsw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, @@ -39,21 +39,21 @@ uint32_t cpu_ldl_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUL | MO_UNALN, mmu_idx); - return cpu_ldl_be_mmu(env, addr, oi, ra); + return cpu_ldl_mmu(env, addr, oi, ra); } uint64_t cpu_ldq_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUQ | MO_UNALN, mmu_idx); - return cpu_ldq_be_mmu(env, addr, oi, ra); + return cpu_ldq_mmu(env, addr, oi, ra); } uint32_t cpu_lduw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUW | MO_UNALN, mmu_idx); - return cpu_ldw_le_mmu(env, addr, oi, ra); + return cpu_ldw_mmu(env, addr, oi, ra); } int cpu_ldsw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, @@ -66,14 +66,14 @@ uint32_t cpu_ldl_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUL | MO_UNALN, mmu_idx); - return cpu_ldl_le_mmu(env, addr, oi, ra); + return cpu_ldl_mmu(env, addr, oi, ra); } uint64_t cpu_ldq_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUQ | MO_UNALN, mmu_idx); - return cpu_ldq_le_mmu(env, addr, oi, ra); + return cpu_ldq_mmu(env, addr, oi, ra); } void cpu_stb_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, @@ -87,42 +87,42 @@ void cpu_stw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUW | MO_UNALN, mmu_idx); - cpu_stw_be_mmu(env, addr, val, oi, ra); + cpu_stw_mmu(env, addr, val, oi, ra); } void cpu_stl_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUL | MO_UNALN, mmu_idx); - cpu_stl_be_mmu(env, addr, val, oi, ra); + cpu_stl_mmu(env, addr, val, oi, ra); } void cpu_stq_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint64_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUQ | MO_UNALN, mmu_idx); - cpu_stq_be_mmu(env, addr, val, oi, ra); + cpu_stq_mmu(env, addr, val, oi, ra); } void cpu_stw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUW | MO_UNALN, mmu_idx); - cpu_stw_le_mmu(env, addr, val, oi, ra); + cpu_stw_mmu(env, addr, val, oi, ra); } void cpu_stl_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUL | MO_UNALN, mmu_idx); - cpu_stl_le_mmu(env, addr, val, oi, ra); + cpu_stl_mmu(env, addr, val, oi, ra); } void cpu_stq_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint64_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUQ | MO_UNALN, mmu_idx); - cpu_stq_le_mmu(env, addr, val, oi, ra); + cpu_stq_mmu(env, addr, val, oi, ra); } /*--------------------------*/ From patchwork Tue May 23 13:47:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684937 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730229wrt; Tue, 23 May 2023 06:52:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4vJYLLOligR6LADVqakqfNNWjJv5BSP4HFnPWpR3SlhbIKGxgnKU4EKZx5SgQx/G8YI5z5 X-Received: by 2002:a05:620a:9481:b0:75b:23a1:3622 with SMTP id sn1-20020a05620a948100b0075b23a13622mr3969569qkn.51.1684849919875; Tue, 23 May 2023 06:51:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849919; cv=none; d=google.com; s=arc-20160816; b=jAo+Jka+NFkTmyh4tzQ4BB7Qa6/QaGxL+NqyjBgrPXA/92T0BUl5xEyNbaLKKMLSrD 3+5Oq82YftN/f92D1ExjzQWTy8FASKxNJTh5WqGXqpuHT1OqsOaLcdQKEUJNHm3l1eGo AsrQeD7uxPUmFXRnV7nm4oVvz9wLZCfMrPZbkJtvE9rYwBdqorMCD2OduQSCXmpK5rh3 b0aLFgofQJf0UWA/FMd2bX1D0G5pDMPu0HRxwFDpsynz5ocKNNn1de2h6lv18YP+ZXXn 55SuZLrFrCxWiFLErwpsGW08PednRtqQj8mGFBx/o3mr9tiMkOfXGRE+ULP3jHEXijw4 eB/w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=v6f7IPBB3UCqneSKtPtJW24fjTyGXh2alDOzIwctERw=; b=ric54AsrVvTgUBrLoktpEJvRuSRSXVVoCJZONk2ZG/vyOjsrDtDNA31IQqZDQZnHO0 iIgptFgD35hoJP2n82Iu7iWiSrTz5MXDfxsI5vTOr0AUMCMDWXtKGBfkSCzVTByNURrd NHkXcub+fXTD2wCpjzPM+qa9zCjBdW1oIYrEu4DmIOKaTLua8EP3qD8Pe3wA2Hauyo2E xErqT6Kt7wuh+ap9fYc/LLh+YsAU9S59EUb9lpxSi96FGTSk19Dpd2WRq3j6LiHXJ0Ib 3dRMPTdA1HnxqeN82cJePWYnUQdDCON9xWdHwWP/zSf0mh1psf3+lZJZnRYT1IvsLFtq N53Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KIC5takI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id pc47-20020a05620a842f00b0075950b9d467si4761954qkn.579.2023.05.23.06.51.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:51:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KIC5takI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMm-0000cJ-0c; Tue, 23 May 2023 09:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMj-0000Wa-6S for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:53 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMh-00018E-9l for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:52 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64d2c865e4eso3974380b3a.0 for ; Tue, 23 May 2023 06:47:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849670; x=1687441670; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v6f7IPBB3UCqneSKtPtJW24fjTyGXh2alDOzIwctERw=; b=KIC5takIMy2ID2zARHaQrgdgQXLsMAJ/7nmjISmktIqXj0o338YBUJHtw2XbslxfKU 9SXe+HzM2PrgCJBjWuOfzwT6RtCR7X1oRPHqon54a/C5XusMXHbx7kj61VUS+Zpy++Fb RqPTGz87FwBTKdnKdgf/oGHOUxkR9hLI1uPtJkj5kFvNO9Zbdy/c6smT3DFenoR8DTLl eZh+JOFi1Kev/uRrcCeru2mVT+gGUl9/Rm0H8+MHmOxFLYxloaSgw8vJXVTk9y6wNVua XJnx7y1nFhEtnbrqNLpRpPb3wXY5phaOE/vMw7WandWshkthfiT851ASnLDmsefcDc0G 3nSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849670; x=1687441670; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v6f7IPBB3UCqneSKtPtJW24fjTyGXh2alDOzIwctERw=; b=NQ6lfInmJv5U1pX88zEqaReSqZ9GOhk01Bp89gMMrPGriT6U28HdBxG4wuo0kXuZFy Pi4iha7iGWH/kwilYhrWrM6QXWEF4nuwxZsXpvheD9l5HM2wBGupyntB3Ylp4m1Xg39l 8NNKX5NKsU3EI84aau2GajNVK+TOXV0O09XDC5dFNNHbKuOJpxZYaNzkZ7Jvkyo46sWN II2aUu5EIWD91IRHO3wW4GUarh2P07FJrzXxKehG+KvYHq2wFMtKOYJGvDgK/bgCu/Qd eppTgVSOO9wr4NjssVj7kOX3F3A/g495eS8Hec2yVi4DFWDYngnLzAN3/v4ZUz0KgGgm Bh7g== X-Gm-Message-State: AC+VfDy1GOzYsOAvJ1RvJVDt0FzHFXFsWf5I8GMlQ+d3AFQiSt5fQxjQ QZpVYcFYuUhtvFRKte3fXUn+IXQvZH/9MwQ7i+o= X-Received: by 2002:a05:6a00:2282:b0:643:b263:404 with SMTP id f2-20020a056a00228200b00643b2630404mr20900312pfe.33.1684849670030; Tue, 23 May 2023 06:47:50 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, qemu-s390x@nongnu.org, David Hildenbrand , Ilya Leoshkevich Subject: [PATCH v2 17/27] target/s390x: Use cpu_{ld,st}*_mmu in do_csst Date: Tue, 23 May 2023 06:47:23 -0700 Message-Id: <20230523134733.678646-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use cpu_ld16_mmu and cpu_st16_mmu to eliminate the special case, and change all of the *_data_ra functions to match. Note that we check the alignment of both compare and store pointers at the top of the function, so MO_ALIGN* may be safely removed from the individual memory operations. Signed-off-by: Richard Henderson Reviewed-by: David Hildenbrand Reviewed-by: Alex Bennée --- Cc: qemu-s390x@nongnu.org Cc: David Hildenbrand Cc: Ilya Leoshkevich --- target/s390x/tcg/mem_helper.c | 66 ++++++++++++++--------------------- 1 file changed, 27 insertions(+), 39 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 0e0d66b3b6..c757612244 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1737,6 +1737,11 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, uint64_t a2, bool parallel) { uint32_t mem_idx = cpu_mmu_index(env, false); + MemOpIdx oi16 = make_memop_idx(MO_TE | MO_128, mem_idx); + MemOpIdx oi8 = make_memop_idx(MO_TE | MO_64, mem_idx); + MemOpIdx oi4 = make_memop_idx(MO_TE | MO_32, mem_idx); + MemOpIdx oi2 = make_memop_idx(MO_TE | MO_16, mem_idx); + MemOpIdx oi1 = make_memop_idx(MO_8, mem_idx); uintptr_t ra = GETPC(); uint32_t fc = extract32(env->regs[0], 0, 8); uint32_t sc = extract32(env->regs[0], 8, 8); @@ -1780,15 +1785,17 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, } } - /* All loads happen before all stores. For simplicity, load the entire - store value area from the parameter list. */ - svh = cpu_ldq_data_ra(env, pl + 16, ra); - svl = cpu_ldq_data_ra(env, pl + 24, ra); + /* + * All loads happen before all stores. For simplicity, load the entire + * store value area from the parameter list. + */ + svh = cpu_ldq_mmu(env, pl + 16, oi8, ra); + svl = cpu_ldq_mmu(env, pl + 24, oi8, ra); switch (fc) { case 0: { - uint32_t nv = cpu_ldl_data_ra(env, pl, ra); + uint32_t nv = cpu_ldl_mmu(env, pl, oi4, ra); uint32_t cv = env->regs[r3]; uint32_t ov; @@ -1801,8 +1808,8 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, ov = cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra); #endif } else { - ov = cpu_ldl_data_ra(env, a1, ra); - cpu_stl_data_ra(env, a1, (ov == cv ? nv : ov), ra); + ov = cpu_ldl_mmu(env, a1, oi4, ra); + cpu_stl_mmu(env, a1, (ov == cv ? nv : ov), oi4, ra); } cc = (ov != cv); env->regs[r3] = deposit64(env->regs[r3], 32, 32, ov); @@ -1811,21 +1818,20 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, case 1: { - uint64_t nv = cpu_ldq_data_ra(env, pl, ra); + uint64_t nv = cpu_ldq_mmu(env, pl, oi8, ra); uint64_t cv = env->regs[r3]; uint64_t ov; if (parallel) { #ifdef CONFIG_ATOMIC64 - MemOpIdx oi = make_memop_idx(MO_TEUQ | MO_ALIGN, mem_idx); - ov = cpu_atomic_cmpxchgq_be_mmu(env, a1, cv, nv, oi, ra); + ov = cpu_atomic_cmpxchgq_be_mmu(env, a1, cv, nv, oi8, ra); #else /* Note that we asserted !parallel above. */ g_assert_not_reached(); #endif } else { - ov = cpu_ldq_data_ra(env, a1, ra); - cpu_stq_data_ra(env, a1, (ov == cv ? nv : ov), ra); + ov = cpu_ldq_mmu(env, a1, oi8, ra); + cpu_stq_mmu(env, a1, (ov == cv ? nv : ov), oi8, ra); } cc = (ov != cv); env->regs[r3] = ov; @@ -1834,27 +1840,19 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, case 2: { - uint64_t nvh = cpu_ldq_data_ra(env, pl, ra); - uint64_t nvl = cpu_ldq_data_ra(env, pl + 8, ra); - Int128 nv = int128_make128(nvl, nvh); + Int128 nv = cpu_ld16_mmu(env, pl, oi16, ra); Int128 cv = int128_make128(env->regs[r3 + 1], env->regs[r3]); Int128 ov; if (!parallel) { - uint64_t oh = cpu_ldq_data_ra(env, a1 + 0, ra); - uint64_t ol = cpu_ldq_data_ra(env, a1 + 8, ra); - - ov = int128_make128(ol, oh); + ov = cpu_ld16_mmu(env, a1, oi16, ra); cc = !int128_eq(ov, cv); if (cc) { nv = ov; } - - cpu_stq_data_ra(env, a1 + 0, int128_gethi(nv), ra); - cpu_stq_data_ra(env, a1 + 8, int128_getlo(nv), ra); + cpu_st16_mmu(env, a1, nv, oi16, ra); } else if (HAVE_CMPXCHG128) { - MemOpIdx oi = make_memop_idx(MO_TE | MO_128 | MO_ALIGN, mem_idx); - ov = cpu_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi, ra); + ov = cpu_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi16, ra); cc = !int128_eq(ov, cv); } else { /* Note that we asserted !parallel above. */ @@ -1876,29 +1874,19 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, if (cc == 0) { switch (sc) { case 0: - cpu_stb_data_ra(env, a2, svh >> 56, ra); + cpu_stb_mmu(env, a2, svh >> 56, oi1, ra); break; case 1: - cpu_stw_data_ra(env, a2, svh >> 48, ra); + cpu_stw_mmu(env, a2, svh >> 48, oi2, ra); break; case 2: - cpu_stl_data_ra(env, a2, svh >> 32, ra); + cpu_stl_mmu(env, a2, svh >> 32, oi4, ra); break; case 3: - cpu_stq_data_ra(env, a2, svh, ra); + cpu_stq_mmu(env, a2, svh, oi8, ra); break; case 4: - if (!parallel) { - cpu_stq_data_ra(env, a2 + 0, svh, ra); - cpu_stq_data_ra(env, a2 + 8, svl, ra); - } else if (HAVE_ATOMIC128) { - MemOpIdx oi = make_memop_idx(MO_TEUQ | MO_ALIGN_16, mem_idx); - Int128 sv = int128_make128(svl, svh); - cpu_atomic_sto_be_mmu(env, a2, sv, oi, ra); - } else { - /* Note that we asserted !parallel above. */ - g_assert_not_reached(); - } + cpu_st16_mmu(env, a2, int128_make128(svl, svh), oi16, ra); break; default: g_assert_not_reached(); From patchwork Tue May 23 13:47:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684920 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728560wrt; Tue, 23 May 2023 06:48:17 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7OW5PES/nPyihZJoxXM0kZb3eURUnoAnhTYv1aLRjIDToKTXqf0LlmYLRmsGrZw6m0hXb6 X-Received: by 2002:ac8:5b09:0:b0:3f6:b823:f2af with SMTP id m9-20020ac85b09000000b003f6b823f2afmr4487296qtw.1.1684849697015; Tue, 23 May 2023 06:48:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849697; cv=none; d=google.com; s=arc-20160816; b=Tna6+p3d6ylDVrgRXcg0LV75gLgl9F63pYh6gWOOE0qzu+qsBFDe3GlxfmwSfFwGH0 kGocVwyn9FcavGtMDbH+5iiB+YsR5azeEhtf/vxaWXLsNBV7IgtAh6il/fHobINK6CDH F9l53pKdzOzZDOCBQahU3HpIxG9SKSIp0cmip9JR7Nd5FJIsKdsbQQml3UFd2u+JT9xN WkPCfyqtWa/897DSig17lZx6wR2tV1IxMMF4egMNeR8BJjlA5+9a83URaDw999IAIj8I 81t6w6qV/oH1jUYtECCe/+iteIXP/3PxOOtrpehoqqxe0suXYr1HKZRVSVhoPYoJlbmw gbyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=HMMLECzBZvLbq8oOai1RMIPgWJmrRTbYQa8uL1NZwrA=; b=Gcm3vNRznqRMeqQdTLL5VSjKtngpqKfxyLrD+vxh9FmFM6LIVFjO2VuMnpfOTGxVnY yl7fbZrtA3XH2jx6anRjCmKgmJ28evokOpkk9qtdZAHwj87lqhRLpNSKDMBJY6GHeTLV 1sDEBEFGGwGBrJXOYrvZg96oJtfK+39KE4i07zvaRwWHSo9mfWBnUeuhOloJiQauEQn/ LVh1onYO8pkE7AU7EzzBtf27p9By5HxBSGawGRQ7tZLqx8Jj9Ep1DV+aiV/MRTCTZPEX IFQ/E+DT+7jg8qrk3Li6I4iBtsSpxzxJ71H3hhCmkuIzlbEzHmEWYLR+vIzg+clBEAbv lCsA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=o3YpeNIK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j16-20020a05622a039000b003f4f445a3f4si5267150qtx.540.2023.05.23.06.48.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=o3YpeNIK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMn-0000iP-3M; Tue, 23 May 2023 09:47:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMk-0000Y4-BY for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:54 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMi-00018Y-1d for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:54 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-64d1a0d640cso4495553b3a.1 for ; Tue, 23 May 2023 06:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849671; x=1687441671; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HMMLECzBZvLbq8oOai1RMIPgWJmrRTbYQa8uL1NZwrA=; b=o3YpeNIKKmZRvW6OjT2jKQc3duTvw+ipPacNIrmqAPJ0RkTDK7hJyppfz1sfth8tYl Ocu5dSHd4vDKvjANuZsp0bfd65Op5ifQMpUos+KjqifDtD6drlCONGA14Sm2ua0x4zrp 8W3fU6h/Vcjt8+8H0OSiNQJEnwYWJNUj92JY1kxEVWmAjDukLnWcTGkmfE6sAyXZm3Yg 9b+fSyi9tM+WROrLdBmoebCBjD6hBRwaosAA2+DFuPKM02v+xaw/O5EJZcxoQL7wlIC/ 4RONZpn4IxymfNHUW55SHVYkKqxYoWDBQQDGI8l/VC8DN4aK2/eZpyFXHenKrM9U3GsP HZoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849671; x=1687441671; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HMMLECzBZvLbq8oOai1RMIPgWJmrRTbYQa8uL1NZwrA=; b=QJu8jwMHpfMY1NCSh+Eh4szVFHbGQEPSajNdBSwuZtoWCWXnUF3GAGt8iPlru7vPQZ 8qqJCW9OqsUKGzkYjuNTWWRWYOTg6NKJUP6wQgMt23PfMS7DfBhCLTGWy6gmcIiaEx0Q vwGlXV2vpETIxYBBzPTW/cB15dfb+L90RcxfbDRCUbltih7l+5X3OBoUb8gzl4lDzyu+ ciw4OsJ7JxDzuHIcYmNlzkRCYTcP2fpxQoSGEOrw9ifMUYm/K3yiFDtxx81WI0ueEpUs h2S/h/BajWnnKzORBoPkgPCVYpy8RF5b9ggAR174UWlNiKgx4UUlT3/AIRkQIeBxFOW5 4AYw== X-Gm-Message-State: AC+VfDxlK2zAzs8AIRJAx9ObGoIl3uU5eAPSamimrA+pD0fbuqcPvsg3 QBicbs+Sv4kqwutr0Jt1Z0XR7k/xQtiA3rU0se4= X-Received: by 2002:a05:6a21:329a:b0:10b:4f58:3fef with SMTP id yt26-20020a056a21329a00b0010b4f583fefmr8903765pzb.2.1684849670804; Tue, 23 May 2023 06:47:50 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, David Hildenbrand Subject: [PATCH v2 18/27] target/s390x: Always use cpu_atomic_cmpxchgl_be_mmu in do_csst Date: Tue, 23 May 2023 06:47:24 -0700 Message-Id: <20230523134733.678646-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Eliminate the CONFIG_USER_ONLY specialization. Reviewed-by: David Hildenbrand Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- target/s390x/tcg/mem_helper.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index c757612244..aa8ec6ba6d 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1800,13 +1800,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, uint32_t ov; if (parallel) { -#ifdef CONFIG_USER_ONLY - uint32_t *haddr = g2h(env_cpu(env), a1); - ov = qatomic_cmpxchg__nocheck(haddr, cv, nv); -#else - MemOpIdx oi = make_memop_idx(MO_TEUL | MO_ALIGN, mem_idx); - ov = cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra); -#endif + ov = cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi4, ra); } else { ov = cpu_ldl_mmu(env, a1, oi4, ra); cpu_stl_mmu(env, a1, (ov == cv ? nv : ov), oi4, ra); From patchwork Tue May 23 13:47:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684927 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729259wrt; Tue, 23 May 2023 06:49:48 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4pYATiHg4OxvlIvhnzqvgDxjQmYSof1YRY9qyOni1pzC/I3lfTLWbgODwceGdGd3/y4Z4F X-Received: by 2002:a05:622a:14d0:b0:3f4:ef5f:b5c7 with SMTP id u16-20020a05622a14d000b003f4ef5fb5c7mr22653116qtx.0.1684849788049; Tue, 23 May 2023 06:49:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849788; cv=none; d=google.com; s=arc-20160816; b=DEm0cYVWqgVuAzX3/1SnEkyTk/hYkbfg2oJaxFBzzsf7DNIq6Nc4ScuF9SYhSD8wBY 8GtaJdycQ3523e0OVPoRJBjFPnAu7CTPRbrloww7PZlliGktoOtsy5rBr4skrw2sgfOW ZDwZV0e0oqSAxClh2YL8Ku5ODm14G+w3egngF3GSCgjT7gBk3qhjsa397cidLf3j22ZX YquzvFAveyXdg5AsRxVDoWwJitZRACJb90cLjI+YDay5atR8eXlaDS7y3GsxfhiLf8Eq ZrAj7wmgZEmOvSXPcTvO1NlK+f1KsHbQecmRvhxhibMh9M7NlLfm5dofnKLNB4EflUko euVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=MPL5Chc7uKNtPykCbIf1bx9tBI7z2B/RniQ4iaiA6MY=; b=F2Hl1SxZFXl/ZuMAIw/IPsJG9n0y9R8bpILKS3l4C91YQkRW2Etznt3STLSgMrquJh sAZZgrrUv5yNzSnrgYoDEHKk750B+Fwd+IHn5J8QkvJXyiCW/edxgE9RCrymC4stU+NY yMOAdaCUJf/583wXr/DWytPSVoMpNjvVERpNRlD6EudTDbB1A336c76C3WnYU4EfMkDK h496AVTwhi8fv82jc3LeEI8PTHp7KjxjCieds48Ayz2xvucw9J1QLaybyMVT17tLeLKz t0IrdSoDFeWnBOIntdR+d1W9xHiBgzxpiaK92ltdV+Hw82eYvn3F95AktvE/6tE6jqTI bmRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PJsAJODP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p13-20020a05622a13cd00b003e4e4300960si5405516qtk.331.2023.05.23.06.49.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:49:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PJsAJODP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMm-0000eS-GE; Tue, 23 May 2023 09:47:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMk-0000YY-OL for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:54 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMi-00012p-Cu for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:54 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-64d577071a6so3550844b3a.1 for ; Tue, 23 May 2023 06:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849671; x=1687441671; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MPL5Chc7uKNtPykCbIf1bx9tBI7z2B/RniQ4iaiA6MY=; b=PJsAJODPha0GBAe98t8LJjJFCsJcYJGhqSpdj8rTUy7jUFQVlh4QVzgBESvFXuXIPw +1MJvxuylEJKJfcR3aoYW4yP3XLT7u6pS4muVIgFUCNKa762ER7D6b6vRpi9NVbkr+ag 0UlhLkkcV7jV8MNW04QPEwqTE1uyIIz1+yagrH98nYwnZ7r10U6mmbGaW5kskv5BB1XL kO42jqxYqQZqnCoR+OgkwRem7wqZKyU+7fSeThvfHHMRXBmXbFEtIr/0OxahunFEhrcq Kqq0wRmK4jnaovepy2Hf40YVvTVego2VtAmn0dnnajOPuwDZwpn7+NaJDJhtpzJXPLfY NN9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849671; x=1687441671; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MPL5Chc7uKNtPykCbIf1bx9tBI7z2B/RniQ4iaiA6MY=; b=cotzFybudU4ay3XrL+Kwbfd60XLe/jiO/6PT83lpdjtIAAhSlwdnBXTN6ICxjpQsOV CNqQ6zCaDslzBFGVupJag2G5PNiOBS8vTiBOB9waEYZpPBTnTI04f7R7dx0jyA8W7HBw 03h3PHJj/rziJGTHIUzXK11YXTR6kh5EAfu0F+Xpr8F1wi/LgBLQCz8spD/HQ3OymcYQ jn8t10COTfjOfykg0pWR0IrmVydjhO71tHJ138X+rhfDPRC2W77wgtPPs06ThvNgY0vk KJUhPw6ARoXfJnOvCZ95wext24rrPv3ZdsegnKqK1FfbYqVshowmpaw9Moa3W2s280an OpTQ== X-Gm-Message-State: AC+VfDxTfZrujtg5VpPcbPs8hQp0SI4gVG1rpOCLyfRgrCIyXNTsyxK9 GN0pHrCB54nZaoPclboF09T+ZnGkB/T7gLBmFNM= X-Received: by 2002:a05:6a00:14c4:b0:643:9ca:3c7d with SMTP id w4-20020a056a0014c400b0064309ca3c7dmr17814113pfu.4.1684849671632; Tue, 23 May 2023 06:47:51 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 19/27] accel/tcg: Remove cpu_atomic_{ld,st}o_*_mmu Date: Tue, 23 May 2023 06:47:25 -0700 Message-Id: <20230523134733.678646-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Atomic load/store of 128-byte quantities is now handled by cpu_{ld,st}16_mmu. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- accel/tcg/atomic_template.h | 61 +++-------------------------------- include/exec/cpu_ldst.h | 9 ------ accel/tcg/atomic_common.c.inc | 14 -------- 3 files changed, 4 insertions(+), 80 deletions(-) diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h index 404a530f7c..30eee9d066 100644 --- a/accel/tcg/atomic_template.h +++ b/accel/tcg/atomic_template.h @@ -87,33 +87,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, return ret; } -#if DATA_SIZE >= 16 -#if HAVE_ATOMIC128 -ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ, retaddr); - DATA_TYPE val; - - val = atomic16_read(haddr); - ATOMIC_MMU_CLEANUP; - atomic_trace_ld_post(env, addr, oi); - return val; -} - -void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_WRITE, retaddr); - - atomic16_set(haddr, val); - ATOMIC_MMU_CLEANUP; - atomic_trace_st_post(env, addr, oi); -} -#endif -#else +#if DATA_SIZE < 16 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { @@ -188,7 +162,7 @@ GEN_ATOMIC_HELPER_FN(smax_fetch, MAX, SDATA_TYPE, new) GEN_ATOMIC_HELPER_FN(umax_fetch, MAX, DATA_TYPE, new) #undef GEN_ATOMIC_HELPER_FN -#endif /* DATA SIZE >= 16 */ +#endif /* DATA SIZE < 16 */ #undef END @@ -220,34 +194,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, return BSWAP(ret); } -#if DATA_SIZE >= 16 -#if HAVE_ATOMIC128 -ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ, retaddr); - DATA_TYPE val; - - val = atomic16_read(haddr); - ATOMIC_MMU_CLEANUP; - atomic_trace_ld_post(env, addr, oi); - return BSWAP(val); -} - -void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_WRITE, retaddr); - - val = BSWAP(val); - atomic16_set(haddr, val); - ATOMIC_MMU_CLEANUP; - atomic_trace_st_post(env, addr, oi); -} -#endif -#else +#if DATA_SIZE < 16 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { @@ -326,7 +273,7 @@ GEN_ATOMIC_HELPER_FN(add_fetch, ADD, DATA_TYPE, new) #undef ADD #undef GEN_ATOMIC_HELPER_FN -#endif /* DATA_SIZE >= 16 */ +#endif /* DATA_SIZE < 16 */ #undef END #endif /* DATA_SIZE > 1 */ diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index fc1d3d9301..5939688f69 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -300,15 +300,6 @@ Int128 cpu_atomic_cmpxchgo_be_mmu(CPUArchState *env, target_ulong addr, Int128 cmpv, Int128 newv, MemOpIdx oi, uintptr_t retaddr); -Int128 cpu_atomic_ldo_le_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -Int128 cpu_atomic_ldo_be_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -void cpu_atomic_sto_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr); -void cpu_atomic_sto_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr); - #if defined(CONFIG_USER_ONLY) extern __thread uintptr_t helper_retaddr; diff --git a/accel/tcg/atomic_common.c.inc b/accel/tcg/atomic_common.c.inc index fe0eea018f..f255c9e215 100644 --- a/accel/tcg/atomic_common.c.inc +++ b/accel/tcg/atomic_common.c.inc @@ -19,20 +19,6 @@ static void atomic_trace_rmw_post(CPUArchState *env, uint64_t addr, qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_RW); } -#if HAVE_ATOMIC128 -static void atomic_trace_ld_post(CPUArchState *env, uint64_t addr, - MemOpIdx oi) -{ - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); -} - -static void atomic_trace_st_post(CPUArchState *env, uint64_t addr, - MemOpIdx oi) -{ - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} -#endif - /* * Atomic helpers callable from TCG. * These have a common interface and all defer to cpu_atomic_* From patchwork Tue May 23 13:47:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684935 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730084wrt; Tue, 23 May 2023 06:51:37 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ57++3iZ9zG/ooqZznKA9CzA4WfbxvDCS05zoakWcPr+5YbiLa9LOFoS/vU1b8FAXTeVIwP X-Received: by 2002:ae9:ed8a:0:b0:75b:23a1:461 with SMTP id c132-20020ae9ed8a000000b0075b23a10461mr4160495qkg.39.1684849896806; Tue, 23 May 2023 06:51:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849896; cv=none; d=google.com; s=arc-20160816; b=LROiQFi+r3s/TBxnOFZEy5JyscgG9D875Ja4CZZxxxpiXdVo4lCjofNroHosMaPd4e fHK2Zr4rcvQ5LMF40IWJks4t2XSQ2Ns6V4kbqPZc132dbBEAp0nQ40kEHQ4zY/dA9pZ8 OwgOP1tPQ6ZY3JYmFgVcNHs8AvXGpc4Bg7AbbaCxSuiRfXH0ux75eRr1e9Z/KQZWnOTY KQ8E8DCi2B6jYx7p3Cph09+WesF5eEk151flKCldiNgjgXKp6lRUQt57MLmErP+Ewv4h EOB8kiiTzWPGRSIyRp99bgx2PjAi7X0CQspnjH5xvWVuf2QpshGE2LnAA5a8F4AGmdio qgNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=b4QbT7Qp39kM9Wky+d26o1epuHAtvA7Ubq3UboPhQe0=; b=DLlatrVTKRrNMpV7/oO7F0f7NFFexfIDw195SWn91I7YIhZ2zbwdzLLowH0LeQLEqn lEPP469BLBWpaW/A4+gMRQ8UVd0mVW90wGRPcq6JunyyFxp7trTe0cVxeaPIlfFV+jXg d8dGn994XB/Mc533TiTx6q3Q+KSlSOJ1BwpHbvo2nezovVz38WJ5zYhVkcZtyFB3E68i BT99PoNWrtV3RshiOT+bRX2XRhwNvtGeQ8tsGlO5aj5F7MiAFGrJLg29xVF3mmcUNwFC Z8Sfq7cgKtBOpIDoSpL9jo0zFil6TsRugYgfn+DpH8tLiT7dKN/tU+5gc56o5IcJ8mxh Ppig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TsWPMQIh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id hr12-20020a05621423cc00b006212497a8d6si5187931qvb.301.2023.05.23.06.51.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:51:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TsWPMQIh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMp-0000vg-5a; Tue, 23 May 2023 09:47:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMm-0000ff-Lw for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:56 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMk-00018r-Aj for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:56 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-64d293746e0so6209131b3a.2 for ; Tue, 23 May 2023 06:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849672; x=1687441672; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b4QbT7Qp39kM9Wky+d26o1epuHAtvA7Ubq3UboPhQe0=; b=TsWPMQIh04GxLiDDjXXSrPzB42UxoRIWG3+fjes7tpLTxP/uRS+7e/ljC23xr2d5LG Gvwzl9hpzGvE6hF2w8xb/z2iI9cS9wwlYQapSymdxAODHyylonAwAMdbtFgNxCgSW2GR nyiKhsG10M77+9TCjzdsjGLzqD5JCCmX3DbjT7N6ZL2dO1go66FX/sSw7/sGKDBHNRCz eZXRLCjWiiQtkbkeYzNJ3cDoqJI/cnCRzxXtJKKWCMaOVUkTsr7xWWIw77FTgPATNvwm EHB6zYkmxCjVw+b3FmLF9s80eawcovO90sunatSreoZ1ciov48p3Cb9IExBoyCRMMhtO KcmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849672; x=1687441672; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b4QbT7Qp39kM9Wky+d26o1epuHAtvA7Ubq3UboPhQe0=; b=Bh/YFlGB4orVJMU4j23bVmpKTc4yRkSaTaQ/FLBtblwpc/qWjC3pZP7lf8OgSVmnPs RcmsMOT+dMs9z937SnKxwMCOkSkSNRu87+pDUb3kjsCbOsW8cOWU3elSxSQzXwKqRWSs pOZ/uC/5Ry8kFP9YneMohoAHl8ilBTkNkWY96KwxP8HDQ7w2sXXmYw1oA00mqb2fZPsK RnDdoibexu2jhQn+PknMfW4Js7lRv/Ov+nE5F/AM0tQgEPKy47IbFstZif1t2dAX0ahI qE++tYnCP262256pptq4nuyizNrf9JyBEPMJLb5BtRocnL5qK4iiTZqd28oiqL9J4/6B dsOA== X-Gm-Message-State: AC+VfDwIL8DG894EZQ3V0XeWzhuNH6zyE8uxnQdYfRPIz2jimSQszBw/ G6uXQ60TqxuD1new6HVyK5S/LNde2NUhHcN0c98= X-Received: by 2002:a05:6a00:15c7:b0:62d:d045:392 with SMTP id o7-20020a056a0015c700b0062dd0450392mr20465988pfu.32.1684849672421; Tue, 23 May 2023 06:47:52 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 20/27] accel/tcg: Remove prot argument to atomic_mmu_lookup Date: Tue, 23 May 2023 06:47:26 -0700 Message-Id: <20230523134733.678646-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Now that load/store are gone, we're always passing PAGE_READ | PAGE_WRITE for RMW atomic operations. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- accel/tcg/atomic_template.h | 32 ++++++-------- accel/tcg/cputlb.c | 85 ++++++++++++++----------------------- accel/tcg/user-exec.c | 8 +--- 3 files changed, 45 insertions(+), 80 deletions(-) diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h index 30eee9d066..e312acd16d 100644 --- a/accel/tcg/atomic_template.h +++ b/accel/tcg/atomic_template.h @@ -73,8 +73,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE cmpv, ABI_TYPE newv, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); DATA_TYPE ret; #if DATA_SIZE == 16 @@ -91,8 +90,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); DATA_TYPE ret; ret = qatomic_xchg__nocheck(haddr, val); @@ -105,9 +103,8 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) \ { \ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - DATA_TYPE ret; \ + DATA_TYPE *haddr, ret; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ ret = qatomic_##X(haddr, val); \ ATOMIC_MMU_CLEANUP; \ atomic_trace_rmw_post(env, addr, oi); \ @@ -137,9 +134,8 @@ GEN_ATOMIC_HELPER(xor_fetch) ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE xval, MemOpIdx oi, uintptr_t retaddr) \ { \ - XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - XDATA_TYPE cmp, old, new, val = xval; \ + XDATA_TYPE *haddr, cmp, old, new, val = xval; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ smp_mb(); \ cmp = qatomic_read__nocheck(haddr); \ do { \ @@ -180,8 +176,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE cmpv, ABI_TYPE newv, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); DATA_TYPE ret; #if DATA_SIZE == 16 @@ -198,8 +193,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); ABI_TYPE ret; ret = qatomic_xchg__nocheck(haddr, BSWAP(val)); @@ -212,9 +206,8 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) \ { \ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - DATA_TYPE ret; \ + DATA_TYPE *haddr, ret; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ ret = qatomic_##X(haddr, BSWAP(val)); \ ATOMIC_MMU_CLEANUP; \ atomic_trace_rmw_post(env, addr, oi); \ @@ -241,9 +234,8 @@ GEN_ATOMIC_HELPER(xor_fetch) ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE xval, MemOpIdx oi, uintptr_t retaddr) \ { \ - XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - XDATA_TYPE ldo, ldn, old, new, val = xval; \ + XDATA_TYPE *haddr, ldo, ldn, old, new, val = xval; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ smp_mb(); \ ldn = qatomic_read__nocheck(haddr); \ do { \ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b1e13d165c..9cb0b697d1 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1896,12 +1896,9 @@ static bool mmu_lookup(CPUArchState *env, target_ulong addr, MemOpIdx oi, /* * Probe for an atomic operation. Do not allow unaligned operations, * or io operations to proceed. Return the host address. - * - * @prot may be PAGE_READ, PAGE_WRITE, or PAGE_READ|PAGE_WRITE. */ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, - MemOpIdx oi, int size, int prot, - uintptr_t retaddr) + MemOpIdx oi, int size, uintptr_t retaddr) { uintptr_t mmu_idx = get_mmuidx(oi); MemOp mop = get_memop(oi); @@ -1937,54 +1934,37 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, tlbe = tlb_entry(env, mmu_idx, addr); /* Check TLB entry and enforce page permissions. */ - if (prot & PAGE_WRITE) { - tlb_addr = tlb_addr_write(tlbe); - if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(env, mmu_idx, index, MMU_DATA_STORE, - addr & TARGET_PAGE_MASK)) { - tlb_fill(env_cpu(env), addr, size, - MMU_DATA_STORE, mmu_idx, retaddr); - index = tlb_index(env, mmu_idx, addr); - tlbe = tlb_entry(env, mmu_idx, addr); - } - tlb_addr = tlb_addr_write(tlbe) & ~TLB_INVALID_MASK; - } - - if (prot & PAGE_READ) { - /* - * Let the guest notice RMW on a write-only page. - * We have just verified that the page is writable. - * Subpage lookups may have left TLB_INVALID_MASK set, - * but addr_read will only be -1 if PAGE_READ was unset. - */ - if (unlikely(tlbe->addr_read == -1)) { - tlb_fill(env_cpu(env), addr, size, - MMU_DATA_LOAD, mmu_idx, retaddr); - /* - * Since we don't support reads and writes to different - * addresses, and we do have the proper page loaded for - * write, this shouldn't ever return. But just in case, - * handle via stop-the-world. - */ - goto stop_the_world; - } - /* Collect TLB_WATCHPOINT for read. */ - tlb_addr |= tlbe->addr_read; - } - } else /* if (prot & PAGE_READ) */ { - tlb_addr = tlbe->addr_read; - if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(env, mmu_idx, index, MMU_DATA_LOAD, - addr & TARGET_PAGE_MASK)) { - tlb_fill(env_cpu(env), addr, size, - MMU_DATA_LOAD, mmu_idx, retaddr); - index = tlb_index(env, mmu_idx, addr); - tlbe = tlb_entry(env, mmu_idx, addr); - } - tlb_addr = tlbe->addr_read & ~TLB_INVALID_MASK; + tlb_addr = tlb_addr_write(tlbe); + if (!tlb_hit(tlb_addr, addr)) { + if (!victim_tlb_hit(env, mmu_idx, index, MMU_DATA_STORE, + addr & TARGET_PAGE_MASK)) { + tlb_fill(env_cpu(env), addr, size, + MMU_DATA_STORE, mmu_idx, retaddr); + index = tlb_index(env, mmu_idx, addr); + tlbe = tlb_entry(env, mmu_idx, addr); } + tlb_addr = tlb_addr_write(tlbe) & ~TLB_INVALID_MASK; } + /* + * Let the guest notice RMW on a write-only page. + * We have just verified that the page is writable. + * Subpage lookups may have left TLB_INVALID_MASK set, + * but addr_read will only be -1 if PAGE_READ was unset. + */ + if (unlikely(tlbe->addr_read == -1)) { + tlb_fill(env_cpu(env), addr, size, MMU_DATA_LOAD, mmu_idx, retaddr); + /* + * Since we don't support reads and writes to different + * addresses, and we do have the proper page loaded for + * write, this shouldn't ever return. But just in case, + * handle via stop-the-world. + */ + goto stop_the_world; + } + /* Collect TLB_WATCHPOINT for read. */ + tlb_addr |= tlbe->addr_read; + /* Notice an IO access or a needs-MMU-lookup access */ if (unlikely(tlb_addr & (TLB_MMIO | TLB_DISCARD_WRITE))) { /* There's really nothing that can be done to @@ -2000,11 +1980,8 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, } if (unlikely(tlb_addr & TLB_WATCHPOINT)) { - QEMU_BUILD_BUG_ON(PAGE_READ != BP_MEM_READ); - QEMU_BUILD_BUG_ON(PAGE_WRITE != BP_MEM_WRITE); - /* therefore prot == watchpoint bits */ - cpu_check_watchpoint(env_cpu(env), addr, size, - full->attrs, prot, retaddr); + cpu_check_watchpoint(env_cpu(env), addr, size, full->attrs, + BP_MEM_READ | BP_MEM_WRITE, retaddr); } return hostaddr; diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 19c2849c21..1e085b1210 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1323,12 +1323,9 @@ uint64_t cpu_ldq_code_mmu(CPUArchState *env, abi_ptr addr, /* * Do not allow unaligned operations to proceed. Return the host address. - * - * @prot may be PAGE_READ, PAGE_WRITE, or PAGE_READ|PAGE_WRITE. */ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, - MemOpIdx oi, int size, int prot, - uintptr_t retaddr) + MemOpIdx oi, int size, uintptr_t retaddr) { MemOp mop = get_memop(oi); int a_bits = get_alignment_bits(mop); @@ -1336,8 +1333,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, /* Enforce guest required alignment. */ if (unlikely(addr & ((1 << a_bits) - 1))) { - MMUAccessType t = prot == PAGE_READ ? MMU_DATA_LOAD : MMU_DATA_STORE; - cpu_loop_exit_sigbus(env_cpu(env), addr, t, retaddr); + cpu_loop_exit_sigbus(env_cpu(env), addr, MMU_DATA_STORE, retaddr); } /* Enforce qemu required alignment. */ From patchwork Tue May 23 13:47:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684921 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728603wrt; Tue, 23 May 2023 06:48:21 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6LcAxkInvOddERjDgRYOyGIUO+k6kxg5YKM+zIPiamb0VJm6sHpm4qjnyEzAQhLgFuEM6h X-Received: by 2002:a05:622a:1449:b0:3f5:31c9:2578 with SMTP id v9-20020a05622a144900b003f531c92578mr21822286qtx.67.1684849701361; Tue, 23 May 2023 06:48:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849701; cv=none; d=google.com; s=arc-20160816; b=zS2bjRWtQ5H0XyXLl3kzMqdXizUHcDImai0BkzexTfUip5ba3lFwt1L4M5WcOOzX00 ICQNXo1aRFVkpDv0MvmkPH6LobEFXazHtWjPcvvjuhj9YyA2k1noxrp+zdS0vXccwjv4 bx+vZ/jljEgBLpzj/PXdNSD2T3RSl4dGCwBEuCc2oxLKzSrYWHuGlxqh365EwPzVDE31 T9GjjJXX8HFGR4aWRGNrz9M85WlGiKVtAA1y1Rx9QgOhDuGk/w5bx5e3EUW+YEG49Bon IbYutvHvtpP9lcXjS4zn93WaVhXWdtyWW8qbzxZPPHjBJuz0PD+COoMGTeTR05XxKjRQ yfmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=xpw127Ta4vkSOaFTONTeoYtJ2r7Ne1NfMPdNp5TPUr4=; b=a7hBMRQYqSe1cFdO/6It09eOrITpHGLZl7kvWkG0PgszP6dWudl+q2ypPCxDzgDsvy fVMnvhtWitouWjKipdL2vNeLyZFZ3vTuKhQXxmFt4/XuEq7eCXzAKxXlrLi7TAuFfYKL shHKTwkk8Ztylb+fIen5DESqV63ABPf4OqsQ2H3IJkl9WwD6ljuSk18M/s6K2APijRFL 5PXHvwDkk1CygjCj5Uj4lT1RTSHPCU30wpf4bbpw+L/QAM+u4FddNSpozcKpsVX+leZQ D6/l7kVW8bxISQRHyx1mdziPoV6JNdTNgP6hBlB2/frbTBE+bVO4wjkY3J9Bx6A+knBV lV/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tssVLQie; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h10-20020a05622a170a00b003f545c24678si4998151qtk.772.2023.05.23.06.48.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:21 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tssVLQie; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMo-0000t8-IO; Tue, 23 May 2023 09:47:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMm-0000dx-7n for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:56 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMk-00019A-GM for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:55 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-64d41763796so2898780b3a.2 for ; Tue, 23 May 2023 06:47:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849673; x=1687441673; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xpw127Ta4vkSOaFTONTeoYtJ2r7Ne1NfMPdNp5TPUr4=; b=tssVLQie36mMRrgMz2fQzeO84WWvyi/xL9+zIaxJClTgvaFW+5E3KKy2vwxFUNOzOk EVOiT/tC9YTPNfPuu4deHC3qxNpXxLtR9UfvtcsoCGgU8jRiMvnoH8USF5tNJjzu0s/T w/9YppwLkAORQB5qMFVMFlg6tEk5c6vVtQWa4SmRv5qzEcXtgeA0ccVLnUytdKCtc+nv JXhATby9QSMoyav/7un9hPe6FEmKfsa4P2w0No3aTJKeihFXhBi4XlDQzHx7MgZDMjpj XMqV+zrTDtzxlrS3fn5t81Rx+/mJWenF0HLdu2QnhENdZYN2xjeQq+t3B6WmDnQHMh2l PokA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849673; x=1687441673; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xpw127Ta4vkSOaFTONTeoYtJ2r7Ne1NfMPdNp5TPUr4=; b=aM72vIqR6ogq5jXG4YpPAw7hn7sI9yOqBggfxpQf5ih5C7H3fVM1nDV+Nmv4si9EXX qCxZjtgGLePha032GvWBWK63vqaFJX0ssIRpp9WWFl/BDq4VUgOnHzDY/sbrFqc57UWF RXnuXuwl6h7ZLQiOoyXnlemDdzzM8JnonK6gshKpCdpUztcjq1quppsn8y/sWKiDFXcV gUyXMbAuo0bQFbDl9KNqWz5jpXwqiKRJUpRNgH/tFxQbxrw8hSCG6yVUFJs+wAXPu9wE Hz7QwVxs7OUe5IWu83xfzZ9ko0wrrM2f91k6dy2hyljxLf8N81EdmqRvCdxqWt6J+n0+ rePA== X-Gm-Message-State: AC+VfDyEIYcLlTMfbK+tKRNjb8dU3lHQHQUrQ/ehr5cWmvdE36rgA/ri PYkSweqFMLMPH2VbGEGugQ32eb81X4Dniw27dNo= X-Received: by 2002:a05:6a00:2484:b0:64d:5c4c:7e20 with SMTP id c4-20020a056a00248400b0064d5c4c7e20mr10703963pfv.26.1684849673220; Tue, 23 May 2023 06:47:53 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 21/27] accel/tcg: Eliminate #if on HAVE_ATOMIC128 and HAVE_CMPXCHG128 Date: Tue, 23 May 2023 06:47:27 -0700 Message-Id: <20230523134733.678646-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org These symbols will shortly become dynamic runtime tests and therefore not appropriate for the preprocessor. Use the matching CONFIG_* symbols for that purpose. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/aarch64/host/atomic128-cas.h | 2 ++ host/include/generic/host/atomic128-ldst.h | 2 +- accel/tcg/cputlb.c | 2 +- accel/tcg/user-exec.c | 2 +- 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/host/include/aarch64/host/atomic128-cas.h b/host/include/aarch64/host/atomic128-cas.h index 80de58e06d..58630107bc 100644 --- a/host/include/aarch64/host/atomic128-cas.h +++ b/host/include/aarch64/host/atomic128-cas.h @@ -37,6 +37,8 @@ static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) return int128_make128(oldl, oldh); } + +# define CONFIG_CMPXCHG128 1 # define HAVE_CMPXCHG128 1 #endif diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index 46911dfb61..06a62e9dd0 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -33,7 +33,7 @@ atomic16_set(Int128 *ptr, Int128 val) } # define HAVE_ATOMIC128 1 -#elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 +#elif defined(CONFIG_CMPXCHG128) && !defined(CONFIG_USER_ONLY) static inline Int128 ATTRIBUTE_ATOMIC128_OPT atomic16_read(Int128 *ptr) { diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 9cb0b697d1..0bd06bf894 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -3038,7 +3038,7 @@ void cpu_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, #include "atomic_template.h" #endif -#if HAVE_CMPXCHG128 || HAVE_ATOMIC128 +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) #define DATA_SIZE 16 #include "atomic_template.h" #endif diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 1e085b1210..dc8d6b5d40 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1371,7 +1371,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, #include "atomic_template.h" #endif -#if HAVE_ATOMIC128 || HAVE_CMPXCHG128 +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) #define DATA_SIZE 16 #include "atomic_template.h" #endif From patchwork Tue May 23 13:47:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684933 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729831wrt; Tue, 23 May 2023 06:51:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ52miZRYN/tK3OkuoXZnPb1o+NkT5DBXjqrUWOumhsYbfa5FAM0+JbRsgzXtBg6KsqCUQ5Q X-Received: by 2002:a37:acf:0:b0:75b:23a1:d854 with SMTP id 198-20020a370acf000000b0075b23a1d854mr4612595qkk.22.1684849860508; Tue, 23 May 2023 06:51:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849860; cv=none; d=google.com; s=arc-20160816; b=RBHI2zioLWT3DDdXpDEzl/0h3jDBuzSBc2JvW6I2lsvWGZtvNs9dI9FBVttUbmD2Qv ftrAQfJ+OVdf6RxJ7uV/QKqn9fbFDfpAfMFtDWlHa54IDU0I49cbwnNrfWEnvXVJH1Dd 8NlF4pRd8ahl0+hJ2CZqPI6Pddpxz7DU1Jw7zWMw8O3s/0xJTJS/uXy0RVwzOLCZwCWi vyYK+89iTNB0G2PERu0F+O8pO2uSFvj5gbmW4Vlm6kijS1Oogi/woEJaauF/WImhMATb J4FEyvPY902O4UhYdPgLLFEy5dYaHl9dTEvqBMhAEEob3kHQ6651B4wxS+0MwcWvuWa6 D2TQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=mquIrAzxY6HIVGUqj9r7egc6IC0VGhEvBfBIbrm5c5Q=; b=BUjjX2IE+9hjFmvVylm3lOAvvxb3YSDlmD7u6Rw4w5wfT4s/nM41ANOxZyvJo/H1AP UYJ5G4DKCZl2ZXkK2bhPd+iHXPok0hPidJrqo62KRQ5HXqPhB64mBSqPWh8jQItC30Xm /10d4vvjgg4DpxzvG/1B3ykZ6+TRsx26HVj9y1ZoN0Ptf/LjvfwmPGGG/UCc8VSCsw61 mpp1MmG0LvOj4XuaksGP590ZUxjShC9An8Etrxing/9FyjM59D884aA2mPCAvEJwS7KS slupedyaM5tfckKpq/m+p4HOynx3oZHrt7xJ9+DXDqI7JKFJLxVkPfUoQlbjhot689mv aLZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="qMyx/P5a"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id gu6-20020a056214260600b006211e921e4fsi5545937qvb.31.2023.05.23.06.51.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:51:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="qMyx/P5a"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMq-00010R-EQ; Tue, 23 May 2023 09:48:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMn-0000ng-Ku for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:57 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMk-000141-Sk for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:57 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-64d44b198baso2519505b3a.0 for ; Tue, 23 May 2023 06:47:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849674; x=1687441674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mquIrAzxY6HIVGUqj9r7egc6IC0VGhEvBfBIbrm5c5Q=; b=qMyx/P5aqqhAZNIMYpm4J+YceY7sMpMhJdOC7k0chnL3I0c6m0NWqYPRl/K6lWcriY xWMG+U5lgWdbcCjwsEUkbbhSxvYYhxYqO82gzGMrXAi1SxzKeO4zi1nXxXMDHeRIntac 2I11kvphGg/Fz/auvrBDs7JfEDKjzzVxaoIwvvNhVc2cAwvY58uMaQvFfjn/KPRtfnqI FRiLYU74M+pe1IOncqzht3WwwptrbfmXJfbfzz/RNPeuROzufB9jdA8nk7BayS52AAnY awSj/Hdh5Sr4GhySaTsr3gnnUEeBkgUACg7QkY9bXG/2KVglkjiwgjWmc2jIB4u7Qbe5 shMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849674; x=1687441674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mquIrAzxY6HIVGUqj9r7egc6IC0VGhEvBfBIbrm5c5Q=; b=BS+orOFaOZYVxIbneO9ZByyeW12LZfDY4eS19MmKciJInDFncbM2eajrsQyiHsEiEG jX2PCUC9xLR+KvIX6ffZ5nBdB8KMa0vzvE8ExbYNnFr17pBUpKPqyOWhFAwSRhWfkCun HS/KAZGjxEg2tJwG2NO8iI3YaTxy1wov0f5IYUX7SxDF1b+aoRfjmPVtMA20XoFelDf5 +Gnes4HHKVQL6ICEQ6trEDm+xxChL7Akl39cDYfvB6qp9yfNT4n8Kz2rWDDQAc4lt+E+ N4UNg7dmMsLadJvLNqsDALn7Kfm9HqTjmIkoAfE1rYHJmD9e+hs/CuBTw6XRtN/7w2c4 M6Kw== X-Gm-Message-State: AC+VfDyiunfFmIxWtm44VrSeMzt0+yK5Afm2+Yoz4V/Ho5L3MWtqFtfY B++8/GJu5sASvRtblLYGSP+/eqVTa+Ru3k1D3tY= X-Received: by 2002:a05:6a00:990:b0:64d:40e0:5580 with SMTP id u16-20020a056a00099000b0064d40e05580mr15394039pfg.3.1684849674059; Tue, 23 May 2023 06:47:54 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 22/27] qemu/atomic128: Split atomic16_read Date: Tue, 23 May 2023 06:47:28 -0700 Message-Id: <20230523134733.678646-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Create both atomic16_read_ro and atomic16_read_rw. Previously we pretended that we had atomic16_read in system mode, because we "know" that all ram is always writable to the host. Now, expose read-only and read-write versions all of the time. For aarch64, do not fall back to __atomic_read_16 even if supported by the compiler, to work around a clang bug. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/aarch64/host/atomic128-ldst.h | 21 ++++++++------- host/include/generic/host/atomic128-ldst.h | 31 ++++++++++++++++------ target/s390x/tcg/mem_helper.c | 2 +- 3 files changed, 36 insertions(+), 18 deletions(-) diff --git a/host/include/aarch64/host/atomic128-ldst.h b/host/include/aarch64/host/atomic128-ldst.h index bd61fce50d..4b1360de39 100644 --- a/host/include/aarch64/host/atomic128-ldst.h +++ b/host/include/aarch64/host/atomic128-ldst.h @@ -11,10 +11,18 @@ #ifndef AARCH64_ATOMIC128_LDST_H #define AARCH64_ATOMIC128_LDST_H -/* Through gcc 10, aarch64 has no support for 128-bit atomics. */ -#if !defined(CONFIG_ATOMIC128) && !defined(CONFIG_USER_ONLY) -/* We can do better than cmpxchg for AArch64. */ -static inline Int128 atomic16_read(Int128 *ptr) +/* + * Through gcc 10, aarch64 has no support for 128-bit atomics. + * Through clang 16, without -march=armv8.4-a, __atomic_load_16 + * is incorrectly expanded to a read-write operation. + */ + +#define HAVE_ATOMIC128_RO 0 +#define HAVE_ATOMIC128_RW 1 + +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); + +static inline Int128 atomic16_read_rw(Int128 *ptr) { uint64_t l, h; uint32_t tmp; @@ -41,9 +49,4 @@ static inline void atomic16_set(Int128 *ptr, Int128 val) : [l] "r"(l), [h] "r"(h)); } -# define HAVE_ATOMIC128 1 -#else -#include "host/include/generic/host/atomic128-ldst.h" -#endif - #endif /* AARCH64_ATOMIC128_LDST_H */ diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index 06a62e9dd0..79d208b7a4 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -12,16 +12,25 @@ #define HOST_ATOMIC128_LDST_H #if defined(CONFIG_ATOMIC128) +# define HAVE_ATOMIC128_RO 1 +# define HAVE_ATOMIC128_RW 1 + static inline Int128 ATTRIBUTE_ATOMIC128_OPT -atomic16_read(Int128 *ptr) +atomic16_read_ro(const Int128 *ptr) { - __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); + const __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r; r.i = qatomic_read__nocheck(ptr_align); return r.s; } +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_read_rw(Int128 *ptr) +{ + return atomic16_read_ro(ptr); +} + static inline void ATTRIBUTE_ATOMIC128_OPT atomic16_set(Int128 *ptr, Int128 val) { @@ -32,10 +41,14 @@ atomic16_set(Int128 *ptr, Int128 val) qatomic_set__nocheck(ptr_align, v.i); } -# define HAVE_ATOMIC128 1 -#elif defined(CONFIG_CMPXCHG128) && !defined(CONFIG_USER_ONLY) +#elif defined(CONFIG_CMPXCHG128) +# define HAVE_ATOMIC128_RO 0 +# define HAVE_ATOMIC128_RW 1 + +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); + static inline Int128 ATTRIBUTE_ATOMIC128_OPT -atomic16_read(Int128 *ptr) +atomic16_read_rw(Int128 *ptr) { /* Maybe replace 0 with 0, returning the old value. */ Int128 z = int128_make64(0); @@ -52,12 +65,14 @@ atomic16_set(Int128 *ptr, Int128 val) } while (int128_ne(old, cmp)); } -# define HAVE_ATOMIC128 1 #else +# define HAVE_ATOMIC128_RO 0 +# define HAVE_ATOMIC128_RW 0 + /* Fallback definitions that must be optimized away, or error. */ -Int128 QEMU_ERROR("unsupported atomic") atomic16_read(Int128 *ptr); +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_rw(Int128 *ptr); void QEMU_ERROR("unsupported atomic") atomic16_set(Int128 *ptr, Int128 val); -# define HAVE_ATOMIC128 0 #endif #endif /* HOST_ATOMIC128_LDST_H */ diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index aa8ec6ba6d..d02ec861d8 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1780,7 +1780,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, max = 3; #endif if ((HAVE_CMPXCHG128 ? 0 : fc + 2 > max) || - (HAVE_ATOMIC128 ? 0 : sc > max)) { + (HAVE_ATOMIC128_RW ? 0 : sc > max)) { cpu_loop_exit_atomic(env_cpu(env), ra); } } From patchwork Tue May 23 13:47:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684938 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730444wrt; Tue, 23 May 2023 06:52:28 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6lpn9VOSWH3JqsQbQEF8FccdO11iWlHQXo1GGFKnmE90Na0SKHhG/zNvsHl3uXxdtnL259 X-Received: by 2002:ac8:7fcc:0:b0:3f6:bbca:f235 with SMTP id b12-20020ac87fcc000000b003f6bbcaf235mr2015341qtk.40.1684849948744; Tue, 23 May 2023 06:52:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849948; cv=none; d=google.com; s=arc-20160816; b=QBbQx1gsPm+3joJkgl+0Qz4ur2ro8BRZ3W7Qy1koQeTsI/IB7rmJ5JOhHI8IVJPoDn pgyXt38+bvYApdQhQPOJX/l9Usn70VpDKYNeP+EWJJXPBbfZj6bUVIrs/IMcWUuyelX6 QbQRzr+9wKTAVDEFTqg9TFR6YhBmNq8XroRpWmlUWj4+4rhqEsLEu8JElNWLGGD9JSQY Aq8ly7RKJJE2wlKA5P4JWLRgOfeHGD30q864kvCsuNjd/8GQQf8/zpJdN7jB5bEQgoaF eJVC2Ey12CYYG8k7QvGsEvTRNecDayJDf+PY0M99pn0nA4Hx1pGWjRRLDj+OAZuZw22u S58g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=HBWDA1+uXlJlZoqOFW38eoLfuSmJeXie3CQojpPMW+I=; b=ZXw/VeISzXPAEY1kS07zddRYMqFNFapCXrLS/FfHr3eXamKD22QUmOQ9fk1JvWUfGE G9FzVrfkT71rDpMaT4FiVeDQ1cvS8wVW3A8gChtKD+6EYfOqOdApHpGbfApwv1bgp2wa M/QvCgKnt97RnHB6Z5Tw2PIgWzEYJywksgxPH647zTLeygUfs/wvCdSQDJZ982qEN4IL QE8YquZgTlgGn0MPaNKtp/mveoLzQxvSYgvZQ4sz8nZfVTmk6pEJn8hr37oqZ125q4GA RSTwGmbJMXoCMo5FNJr67GXpBXvCJriXWQri50H62WjeNfQRFCl0BuHuWVDWZM8auFXk QPHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N5MnVjaJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 11-20020ac8590b000000b003f3951fc9d3si5360104qty.133.2023.05.23.06.52.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:52:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N5MnVjaJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMs-000187-9w; Tue, 23 May 2023 09:48:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMo-0000sd-BY for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:58 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMm-00019b-BO for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:58 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d2a87b9daso3894082b3a.0 for ; Tue, 23 May 2023 06:47:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849675; x=1687441675; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HBWDA1+uXlJlZoqOFW38eoLfuSmJeXie3CQojpPMW+I=; b=N5MnVjaJ4wrouWOmhjaaU2EhfE5dsyW2kKm7ELAGh4vVo2mAJiSHiXdmpLQPbVwnXX 1ZiKJNiCmDQkdNoYOYwyxd/pOIYb7o3gRAkMpxqOOc6zt4lqiRwafDO8kDYDTMQzxYHX MTgjGXG6/SIQp6mV0R2EikJwZQhPGX993IRpAE8pml/DY5vuV2yYqFKXO9ba0/CBrJbN hlL8kWKqnZT8KqwEnOAQU2LVqDajO3yyC1Qi/kQ6puzMIlcTOD9xJNHR0g/cBlJekRDk 17602pnP2YG/svp884GF0OpfQyDXyyqJQO1ZXJyZmPyB9BXrduTA0PlV/ojSKrkIL9RT B1hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849675; x=1687441675; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HBWDA1+uXlJlZoqOFW38eoLfuSmJeXie3CQojpPMW+I=; b=d2Dl/sCo+7afrCe8MBzuqRcJOVqFnZ9usCtFsJrFjD9dTIno4mQzSJ+uh85sr2YVPx Em3kkQ5VAvWcwX5zuFHeyMsRnAZUPhoHvtDG1H5IfvlWmDm/sSOSbshgBvUSLmBXqCU+ 9kuD981Sx+zTRLNZ9giz2qSPlvqly7nxKQh7q3x/3y+HwNJpIVju2+V2Mwknfe35pOlj MqmYo0ijC67gp4daLke36riPPrdeO576j20AmuchPDvLSb8sPEULvlYUcWqqMOxwXBOZ SHnxRr2hexgG2+NFkzNCnfCTECmC9mSEp9VMRvcHVr4R+3l2ZmRcJG6MsMC34OIb8yBv XHzg== X-Gm-Message-State: AC+VfDzSSUfmN5YUv0EPe5Vr7ENKLJq2XvnYeqSAuZkxSJng4swyj1Io DLH+om5w4ZTK4bvYaLFiSDMIdk+eNtBspuHUknI= X-Received: by 2002:a05:6a00:124b:b0:64d:742f:f590 with SMTP id u11-20020a056a00124b00b0064d742ff590mr6967413pfi.8.1684849674963; Tue, 23 May 2023 06:47:54 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 23/27] accel/tcg: Correctly use atomic128.h in ldst_atomicity.c.inc Date: Tue, 23 May 2023 06:47:29 -0700 Message-Id: <20230523134733.678646-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Remove the locally defined load_atomic16 and store_atomic16, along with HAVE_al16 and HAVE_al16_fast in favor of the routines defined in atomic128.h. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- accel/tcg/cputlb.c | 2 +- accel/tcg/ldst_atomicity.c.inc | 118 +++++++-------------------------- 2 files changed, 24 insertions(+), 96 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 0bd06bf894..90c72c9940 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2712,7 +2712,7 @@ static uint64_t do_st16_leN(CPUArchState *env, MMULookupPageData *p, case MO_ATOM_WITHIN16_PAIR: /* Since size > 8, this is the half that must be atomic. */ - if (!HAVE_al16) { + if (!HAVE_ATOMIC128_RW) { cpu_loop_exit_atomic(env_cpu(env), ra); } return store_whole_le16(p->haddr, p->size, val_le); diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index b89631bbef..0f6b3f8ab6 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -16,18 +16,6 @@ #endif #define HAVE_al8_fast (ATOMIC_REG_SIZE >= 8) -#if defined(CONFIG_ATOMIC128) -# define HAVE_al16_fast true -#else -# define HAVE_al16_fast false -#endif -#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) -# define HAVE_al16 true -#else -# define HAVE_al16 false -#endif - - /** * required_atomicity: * @@ -146,26 +134,6 @@ static inline uint64_t load_atomic8(void *pv) return qatomic_read__nocheck(p); } -/** - * load_atomic16: - * @pv: host address - * - * Atomically load 16 aligned bytes from @pv. - */ -static inline Int128 ATTRIBUTE_ATOMIC128_OPT -load_atomic16(void *pv) -{ -#ifdef CONFIG_ATOMIC128 - __uint128_t *p = __builtin_assume_aligned(pv, 16); - Int128Alias r; - - r.u = qatomic_read__nocheck(p); - return r.s; -#else - qemu_build_not_reached(); -#endif -} - /** * load_atomic8_or_exit: * @env: cpu context @@ -211,8 +179,8 @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv) { Int128 *p = __builtin_assume_aligned(pv, 16); - if (HAVE_al16_fast) { - return load_atomic16(p); + if (HAVE_ATOMIC128_RO) { + return atomic16_read_ro(p); } #ifdef CONFIG_USER_ONLY @@ -232,14 +200,9 @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv) * In system mode all guest pages are writable, and for user-only * we have just checked writability. Try cmpxchg. */ -#if defined(CONFIG_CMPXCHG128) - /* Swap 0 with 0, with the side-effect of returning the old value. */ - { - Int128Alias r; - r.u = __sync_val_compare_and_swap_16((__uint128_t *)p, 0, 0); - return r.s; + if (HAVE_ATOMIC128_RW) { + return atomic16_read_rw(p); } -#endif /* Ultimate fallback: re-execute in serial context. */ cpu_loop_exit_atomic(env_cpu(env), ra); @@ -360,11 +323,10 @@ static uint64_t load_atom_extract_al16_or_exit(CPUArchState *env, uintptr_t ra, static inline uint64_t ATTRIBUTE_ATOMIC128_OPT load_atom_extract_al16_or_al8(void *pv, int s) { -#if defined(CONFIG_ATOMIC128) uintptr_t pi = (uintptr_t)pv; int o = pi & 7; int shr = (HOST_BIG_ENDIAN ? 16 - s - o : o) * 8; - __uint128_t r; + Int128 r; pv = (void *)(pi & ~7); if (pi & 8) { @@ -373,18 +335,14 @@ load_atom_extract_al16_or_al8(void *pv, int s) uint64_t b = qatomic_read__nocheck(p8 + 1); if (HOST_BIG_ENDIAN) { - r = ((__uint128_t)a << 64) | b; + r = int128_make128(b, a); } else { - r = ((__uint128_t)b << 64) | a; + r = int128_make128(a, b); } } else { - __uint128_t *p16 = __builtin_assume_aligned(pv, 16, 0); - r = qatomic_read__nocheck(p16); + r = atomic16_read_ro(pv); } - return r >> shr; -#else - qemu_build_not_reached(); -#endif + return int128_getlo(int128_urshift(r, shr)); } /** @@ -472,7 +430,7 @@ static uint16_t load_atom_2(CPUArchState *env, uintptr_t ra, if (likely((pi & 1) == 0)) { return load_atomic2(pv); } - if (HAVE_al16_fast) { + if (HAVE_ATOMIC128_RO) { return load_atom_extract_al16_or_al8(pv, 2); } @@ -511,7 +469,7 @@ static uint32_t load_atom_4(CPUArchState *env, uintptr_t ra, if (likely((pi & 3) == 0)) { return load_atomic4(pv); } - if (HAVE_al16_fast) { + if (HAVE_ATOMIC128_RO) { return load_atom_extract_al16_or_al8(pv, 4); } @@ -557,7 +515,7 @@ static uint64_t load_atom_8(CPUArchState *env, uintptr_t ra, if (HAVE_al8 && likely((pi & 7) == 0)) { return load_atomic8(pv); } - if (HAVE_al16_fast) { + if (HAVE_ATOMIC128_RO) { return load_atom_extract_al16_or_al8(pv, 8); } @@ -607,8 +565,8 @@ static Int128 load_atom_16(CPUArchState *env, uintptr_t ra, * If the host does not support 16-byte atomics, wait until we have * examined the atomicity parameters below. */ - if (HAVE_al16_fast && likely((pi & 15) == 0)) { - return load_atomic16(pv); + if (HAVE_ATOMIC128_RO && likely((pi & 15) == 0)) { + return atomic16_read_ro(pv); } atmax = required_atomicity(env, pi, memop); @@ -687,36 +645,6 @@ static inline void store_atomic8(void *pv, uint64_t val) qatomic_set__nocheck(p, val); } -/** - * store_atomic16: - * @pv: host address - * @val: value to store - * - * Atomically store 16 aligned bytes to @pv. - */ -static inline void ATTRIBUTE_ATOMIC128_OPT -store_atomic16(void *pv, Int128Alias val) -{ -#if defined(CONFIG_ATOMIC128) - __uint128_t *pu = __builtin_assume_aligned(pv, 16); - qatomic_set__nocheck(pu, val.u); -#elif defined(CONFIG_CMPXCHG128) - __uint128_t *pu = __builtin_assume_aligned(pv, 16); - __uint128_t o; - - /* - * Without CONFIG_ATOMIC128, __atomic_compare_exchange_n will always - * defer to libatomic, so we must use __sync_*_compare_and_swap_16 - * and accept the sequential consistency that comes with it. - */ - do { - o = *pu; - } while (!__sync_bool_compare_and_swap_16(pu, o, val.u)); -#else - qemu_build_not_reached(); -#endif -} - /** * store_atom_4x2 */ @@ -957,7 +885,7 @@ static uint64_t store_whole_le16(void *pv, int size, Int128 val_le) int sh = o * 8; Int128 m, v; - qemu_build_assert(HAVE_al16); + qemu_build_assert(HAVE_ATOMIC128_RW); /* Like MAKE_64BIT_MASK(0, sz), but larger. */ if (sz <= 64) { @@ -1017,7 +945,7 @@ static void store_atom_2(CPUArchState *env, uintptr_t ra, return; } } else if ((pi & 15) == 7) { - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { Int128 v = int128_lshift(int128_make64(val), 56); Int128 m = int128_lshift(int128_make64(0xffff), 56); store_atom_insert_al16(pv - 7, v, m); @@ -1086,7 +1014,7 @@ static void store_atom_4(CPUArchState *env, uintptr_t ra, return; } } else { - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { store_whole_le16(pv, 4, int128_make64(cpu_to_le32(val))); return; } @@ -1151,7 +1079,7 @@ static void store_atom_8(CPUArchState *env, uintptr_t ra, } break; case MO_64: - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { store_whole_le16(pv, 8, int128_make64(cpu_to_le64(val))); return; } @@ -1177,8 +1105,8 @@ static void store_atom_16(CPUArchState *env, uintptr_t ra, uint64_t a, b; int atmax; - if (HAVE_al16_fast && likely((pi & 15) == 0)) { - store_atomic16(pv, val); + if (HAVE_ATOMIC128_RW && likely((pi & 15) == 0)) { + atomic16_set(pv, val); return; } @@ -1206,7 +1134,7 @@ static void store_atom_16(CPUArchState *env, uintptr_t ra, } break; case -MO_64: - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { uint64_t val_le; int s2 = pi & 15; int s1 = 16 - s2; @@ -1233,8 +1161,8 @@ static void store_atom_16(CPUArchState *env, uintptr_t ra, } break; case MO_128: - if (HAVE_al16) { - store_atomic16(pv, val); + if (HAVE_ATOMIC128_RW) { + atomic16_set(pv, val); return; } break; From patchwork Tue May 23 13:47:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684929 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729405wrt; Tue, 23 May 2023 06:50:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5kqPWV5WKc5QLwe0TqnoHuTMixXq5zcVcO1OFapickL3bpXnAyMSBLTTTP9wP9ramzPBS9 X-Received: by 2002:a37:c16:0:b0:75b:23a1:d846 with SMTP id 22-20020a370c16000000b0075b23a1d846mr4484346qkm.8.1684849803905; Tue, 23 May 2023 06:50:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849803; cv=none; d=google.com; s=arc-20160816; b=sDI+teZxC6u4gWpZNMqqgNtPSqGlrABM8804UH7M8zHwsOWdDMI5GBlG4EFq+iMqoG +9JbPd9dcsH0AFjAbR+u5DX9j4NM8SxkC7r122zqiA+Jd9ZAS5gaQ4TEVbAMqiBIfVpC e7wKiVe+qQnN89/34oeX4Ufh4Nb/fxO3b0AU1G0/XLzDijOoUmcahwDWwzzVE6t052BX aKMrdRIn5N6dvYWFGptuOM7segy0W7yKB3Z+iC1tDvxxAflUdAegbCYpQAZD6QN7z+KL r3s/izD2yFKGPd7yBrptBzb0vCnDhZpH5C5rUo0apif3oK7dM4R5M65qlggIzE8I2a5m ZKiw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=2/tTPPnUnjG19dY5X021rmXFRG/giw/axI09dO2DG2Y=; b=KDlombvxof5IhRgQLNAEUU4zXti+FpdaCF3CCBWjI4cyrhDIE4Zn8vl1OnLubq7Vay COmTw4GYUpBwL6YUT2BXS88CZKDdfqhU6uHKQO4XdbuwvSm3zfj26X/RD5Eo8JHNe1xW 0wobzX0BpH1NP4J3NuS+yCLZEE8O1AgG1sLVI1S0id8nqhjUPBYunb/p226CfVyaLox+ 6WoAVGOQiFxkLdV/UTKQYW4+LFDLxUJe//Iv2i/Ijb1rSbZ7ldkKTid4UnGlsoLKNwhZ OAFEgAS6exuWmWfNVP4gJtIZu3F1f7Gzmvr8NGjSvoq6YH03Df+seRkAcokxoag0sJNy h6Fg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Cv/xlKbw"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id iy7-20020a0562140f6700b006239c39fcabsi5345676qvb.540.2023.05.23.06.50.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:50:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Cv/xlKbw"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMr-000151-4i; Tue, 23 May 2023 09:48:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMq-0000zB-1T for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:00 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMn-00019t-9g for qemu-devel@nongnu.org; Tue, 23 May 2023 09:47:59 -0400 Received: by mail-pf1-x435.google.com with SMTP id d2e1a72fcca58-64d44b198baso2519528b3a.0 for ; Tue, 23 May 2023 06:47:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849676; x=1687441676; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2/tTPPnUnjG19dY5X021rmXFRG/giw/axI09dO2DG2Y=; b=Cv/xlKbwfHDjFIVEvo52S3jsphYvFbzTMmeei6lnkN1/zTl8n+VjGvVmFw0C3hH0lN AAb+m2pmrpT3Tw7By4VlSWWF2g7QUIiAJxbBojtVNjKYNpJHj/zS9U4XyT73gpyod7F5 p5JET3zAq+U36ZFmbd1sv5kNiSQkSF4pBzCf3ny97KTJNUzNcTRcfVQnao1MM0n6FicC FEBKdQ4Oj7fCpzHJjUbnOjSWRsyTyQcFsNr99Fplmk67WtbgTfzvc4Wh5aNYHI1rQuCU hp16a50scpkrtRFItQu1a4IXQ/iu52EjAg7c+AnOoBQoH5kOEocZxCdb5XdHeUESmYz2 xLFQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849676; x=1687441676; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2/tTPPnUnjG19dY5X021rmXFRG/giw/axI09dO2DG2Y=; b=AnMYWUqkWybc0X4NNh9fX9KqbbH5EMJYAXhN83S2dZwIpNRptjZUGyD62r1WOTtfth 2fXaVi93QffDfpIvkv12RDn9Dh3VwFBwntQxmIf/4ksXkCzn/6DM72t1r6S3XaNIOwOg 2123mtkuf0MW1gvM7H1vRQYMTT6Utqd9RfJOLrF/d+LFkW5zlMm2YBfb5ou/cl5dMF0e 7xFtDEkLEbP08yj1iPqEMYqvJ7AZk6NEdpLxjE+SMRvxn1fKtIWGTLCCbh3OYoi3JLUJ T1YRxIlq+SCfPMlidkkLB2x03iAwF1jvZ4zmLDTbpC6qgKbS/YTjOMxxV1o6sXuVJgTJ KcCg== X-Gm-Message-State: AC+VfDzJFBrnFXZ3KnP+ZjXa7es5/+/ifedxgv4DbsB3kDsRX7BA2vI5 Uctzn/GA3iaoZEbRtVNPCn0YJERkYc6leo6MSAA= X-Received: by 2002:a05:6a00:1d1d:b0:64d:5683:1977 with SMTP id a29-20020a056a001d1d00b0064d56831977mr8699175pfx.9.1684849675784; Tue, 23 May 2023 06:47:55 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org, =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PATCH v2 24/27] tcg: Split out tcg/debug-assert.h Date: Tue, 23 May 2023 06:47:30 -0700 Message-Id: <20230523134733.678646-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- include/tcg/debug-assert.h | 17 +++++++++++++++++ include/tcg/tcg.h | 9 +-------- MAINTAINERS | 1 + 3 files changed, 19 insertions(+), 8 deletions(-) create mode 100644 include/tcg/debug-assert.h diff --git a/include/tcg/debug-assert.h b/include/tcg/debug-assert.h new file mode 100644 index 0000000000..596765a3d2 --- /dev/null +++ b/include/tcg/debug-assert.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Define tcg_debug_assert + * Copyright (c) 2008 Fabrice Bellard + */ + +#ifndef TCG_DEBUG_ASSERT_H +#define TCG_DEBUG_ASSERT_H + +#if defined CONFIG_DEBUG_TCG || defined QEMU_STATIC_ANALYSIS +# define tcg_debug_assert(X) do { assert(X); } while (0) +#else +# define tcg_debug_assert(X) \ + do { if (!(X)) { __builtin_unreachable(); } } while (0) +#endif + +#endif diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index cd6327b175..072c35f7f5 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -34,6 +34,7 @@ #include "tcg/tcg-mo.h" #include "tcg-target.h" #include "tcg/tcg-cond.h" +#include "tcg/debug-assert.h" /* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 @@ -222,14 +223,6 @@ typedef uint64_t tcg_insn_unit; /* The port better have done this. */ #endif - -#if defined CONFIG_DEBUG_TCG || defined QEMU_STATIC_ANALYSIS -# define tcg_debug_assert(X) do { assert(X); } while (0) -#else -# define tcg_debug_assert(X) \ - do { if (!(X)) { __builtin_unreachable(); } } while (0) -#endif - typedef struct TCGRelocation TCGRelocation; struct TCGRelocation { QSIMPLEQ_ENTRY(TCGRelocation) next; diff --git a/MAINTAINERS b/MAINTAINERS index 1b6466496d..a1b99a31df 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -157,6 +157,7 @@ F: include/exec/helper*.h F: include/sysemu/cpus.h F: include/sysemu/tcg.h F: include/hw/core/tcg-cpu-ops.h +F: include/tcg/ FPU emulation M: Aurelien Jarno From patchwork Tue May 23 13:47:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684923 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1728678wrt; Tue, 23 May 2023 06:48:29 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5M5la4dBqp+G9VA8uRzjSXbKQdOBSXHaW78d6TBITB/AxNKrDUbFOIqInAZvO/voUM8bhZ X-Received: by 2002:a05:622a:184:b0:3f4:fa05:ff61 with SMTP id s4-20020a05622a018400b003f4fa05ff61mr23397094qtw.59.1684849709423; Tue, 23 May 2023 06:48:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849709; cv=none; d=google.com; s=arc-20160816; b=sozsjUqBnr71rS0ugPiA/JRrLgPaCJtjWRkyuVnKyJ/QNIgt7IWKZUuDA8Y7NYkI8A vNfBm6wC1QS/XIP2MzS10Ne3/PtORttB6XhYRbgrdGJpYGGqrrX9N/i2O7V5/U0bwk42 9OTXKJFe80065OHnk5HYlndSmHBueROzChiX0h8IYnWHnR52e8TOzW0gZ6/VupAKjMXZ eGUPXN0n87GHtCF25a6FU6Zw2/1x6mQzx4uopobo5PBWlSw3cw9hfG1JoSocB+xKQLiA 9POukvGYniWWdyfiMFB544DcFmpFtxydsc8MD9Xn5FNllPAn17jaclhf/HLzjgsntr0z wLXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=6WXDf+ZM8xKRR3ok9Sko5lUMigoP5WIzPelyBaHT6/A=; b=WurQrU6OnSuOeCTQXj/n5SEm98qV22mTZwF086eg5yvaqy7WAl2jyXPIph2VN1CDqv QWgy88pizkDJDsMlO0mRa/l/1mtpHR3p+zhc9tq5y7z7wXqny8cZOWkSJUUZBYEf56p6 SiUWi/K3Iz6LkEqjxY2aDXd5X20u4/4xGRt9xjphe25VX0b6nrXi4ssw9KScljMROSty 0JVB75uSK8EVj0ciyUUkfZmIWaMpIUfAfe4aGS4j6UugCpVzOc5a+aIqwuSXltxcirhL 2weWNAAgvQUOAxTyu7ANEj6MI2z35L3p78zbup6Tb+KU9G3PchRexyBdqZ8QnP3flAO7 OeiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LHzBkzOP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h9-20020a0562140da900b005ef56e4f61esi5227474qvh.90.2023.05.23.06.48.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:48:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LHzBkzOP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMs-00019b-SH; Tue, 23 May 2023 09:48:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMq-00010Q-Bx for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:00 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMn-00016c-RI for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:00 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-64d5b4c3ffeso2141962b3a.2 for ; Tue, 23 May 2023 06:47:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849677; x=1687441677; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6WXDf+ZM8xKRR3ok9Sko5lUMigoP5WIzPelyBaHT6/A=; b=LHzBkzOPg1fzTGbJrq0pBhfshleqDI0+hu8H7iD9HEW+ugZuYghdycyGcJQjySw1jK 2zA8HTOdLgfHB9JKYMCXrdzWlGNW3wT4PKLawoMFFmRXuRExFMcS6jwm2H3qwQCJfTqm thELLd3qTDWHwkcGAGZ8SSHtRPqD4RxwILVrrYppBAKMeBvZfkJCsQGIyRXzr3TH22Lz 8dh0/J4BJw27MVs/Ghj4Xo85E580kBg5HIqqIfsOEd7q+NgdTgZmbSsIFOjEfMC2AbK7 QweU1tILIy3CnluGMoFti2zdWgxwosEys0DQTrgq+YNsqmd0x8MVCaPQzNbIbjMXt3z0 OeHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849677; x=1687441677; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6WXDf+ZM8xKRR3ok9Sko5lUMigoP5WIzPelyBaHT6/A=; b=MiN47GiT+3cM9yy95J8zrUyhDZqoMLrhpshRFou/vuVF97GykF8kQJjLYdqZ02/Qew SkWxO6/yHAKUiFoYjzx8SZskUAXpZ0PIJzNnuFDbI2QMnM/fodAtNb7fdiM/FChQc6rO JQkapiIulpZP4wsq4wROkWphPDG8uAeupO9KilflqEpMs9HUoW1YHQSt7DJeqeMZZev5 rhhq4gNXpEgoZrgDJaCznUbvW9IEnTL/L7dHdFiIAA1gTYmNron3/T1Fa5jMHFfRckTk jkvWtZSTLnlgY7OZSTsEWjCt+s1rnIPeWVP5n6n39eRtDT+CjxGXZWB8+ZZFlWkkqnTZ 2k+A== X-Gm-Message-State: AC+VfDx5c4gKuKabP04OLp28RffN2m0DOTbYtzfSrKi29XzaqTrshQ1w 2WS5snjfGMs8x5cjXl2wCSwJUB80cQKW1NmLhXc= X-Received: by 2002:a05:6a00:804:b0:64c:c841:4e8a with SMTP id m4-20020a056a00080400b0064cc8414e8amr19532639pfk.22.1684849676811; Tue, 23 May 2023 06:47:56 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 25/27] qemu/atomic128: Improve cmpxchg fallback for atomic16_set Date: Tue, 23 May 2023 06:47:31 -0700 Message-Id: <20230523134733.678646-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use __sync_bool_compare_and_swap_16 to control the loop, rather than a separate comparison. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/generic/host/atomic128-ldst.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index 79d208b7a4..80fff0643a 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -58,11 +58,14 @@ atomic16_read_rw(Int128 *ptr) static inline void ATTRIBUTE_ATOMIC128_OPT atomic16_set(Int128 *ptr, Int128 val) { - Int128 old = *ptr, cmp; + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); + __int128_t old; + Int128Alias new; + + new.s = val; do { - cmp = old; - old = atomic16_cmpxchg(ptr, cmp, val); - } while (int128_ne(old, cmp)); + old = *ptr_align; + } while (!__sync_bool_compare_and_swap_16(ptr_align, old, new.i)); } #else From patchwork Tue May 23 13:47:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684939 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1730473wrt; Tue, 23 May 2023 06:52:32 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ79gE8kVGv9cU2q2phMu3QVGDqNqRks3eMk5xbHNqQnkkOFPTe4YCFCuUvkZp2XhuD90z6C X-Received: by 2002:a05:622a:40e:b0:3f4:400d:5249 with SMTP id n14-20020a05622a040e00b003f4400d5249mr22853624qtx.67.1684849952251; Tue, 23 May 2023 06:52:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849952; cv=none; d=google.com; s=arc-20160816; b=WiDbN3h3LyThcjOEQP3riBxD+odpnWi/2ahlu2niyx11chtbMJe3L5eYNXVJfksmb5 mZX+uD0F3F+qR0WS4GuiF0xbkYJ8e0JHmQeB1bq23+85yKuM6asMV8xZkdzOYmZ6uJr1 CYKp1cl0SGC6AkJF0GS7GzycCzEXfTjPssJNIaALfpTsB506Ofx0qtm0AeRU9YCACTzg ehee+mnffEtdJtgY78kFHZG1PxXqpHfPq4jmRCfaIDtRhzbuweCU26jnmPNxkzsb0DHt ckJHp9upjrCNB1qKsJ6xYF4l02Ks52mlp9rDc7YqltJkIfgZA6iH72zRyTNA4fZ6kNFn SexQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=PPrkOt+CaBTbeGyLx8wQPnc67J39LOMdRfd1HdHQv5I=; b=ENPNaqolCU/7is7WlNaCUz/b8MBAB8jGIvWBKmx9wD00nf3VpLZo0FPTv5IYWlUsSE tP1RCApjIalvcnVtD03CKLK82IZVRVrubyVl09GIWgQdOLxkAj4WTYVM03plm0MIYJKM OpVVk+iu/Ibbxb+snTlEmMbr8+jNm3hEQrKAipfrK8DL+oW5Vu2lTE2p8CdxnlEGajO5 YKPwj2teHd2UVFwRVTmE4v8GEF5eLSoe7UoV361LzUNO0ALgUgkVQ7dVU/q6BqHcbGll jIpY/OwrcS92JMqYmCkw+hKZvdrQKiK/HRd1s19U2jLjCI5+oqV8zq8C8Yjb66izN7Dv xpew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=n67asMjn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d1-20020a05622a100100b003e8846087bfsi5279816qte.131.2023.05.23.06.52.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:52:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=n67asMjn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMt-00019m-4m; Tue, 23 May 2023 09:48:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMr-00016R-GX for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:01 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMp-0001AJ-Pw for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:01 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-64d3491609fso3270231b3a.3 for ; Tue, 23 May 2023 06:47:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849677; x=1687441677; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PPrkOt+CaBTbeGyLx8wQPnc67J39LOMdRfd1HdHQv5I=; b=n67asMjnfKzgaIwCTyugaT9SJPx/v1B2fNo8xpIO9u6xO98HhfagOFClpAUsy2ofZ7 7tvRiAvbydYUmz6P66//kS1ojGjz3Hb5oF2nNurKyJgR5UXI61F1Jy0xVU9QlWhsI5Be MBUsyG/9643JXa1x02F6fKjJz3QVSFSDxG8G4XP5/9V0akEmK3kFNhSlixEuwPxwsHX/ u7+HgSwMAfmcltZaCsvw82ZBB7bNbkFibA0wkBAs9GguVSKUb9pd9fScaKoorajUiJzL 5Qe4rjKEMAwmrTFB9IpdiBjyU6KDTaA1IHokJiZ60CavhlZb8+bX8y6y2wqnH/leYc/4 8Zug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849677; x=1687441677; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PPrkOt+CaBTbeGyLx8wQPnc67J39LOMdRfd1HdHQv5I=; b=Au5I9avbbIrV68MPaxsWJWHXeSUetGjsLQAMdK6H4C6f/a4SjF7RQyHm5eF78nQwBu zO2v8awBlxmcQ1uXza32mWwaS2Qo373WxTKvY8Q5bGLCrMYHW31Z98xAUzg9UPwdx57i 3KRRZzNGaSp5LTHDaAe/ABFwo2x2Bw6+ZjN/l2Xs37kRc1MnltzKiX2nYlb1y1eGKdNO 9a48IEjBrjpuo0PR1bbdsCMMhIgO1cUh9zyVsIRO+B/3D48v7vV8x/Cpf1Qc9BU9CJ/a OPW3yVF45SahTHA1Npgvxy5008KR8GWVCrl4T5/TS9mGBCMoY/z4lzp1h1IgjcUUtjkf L9iA== X-Gm-Message-State: AC+VfDwkSPASUI5ZbHV768pwqL+JL+59jWf+RMMMlSRBm0heEgazMQ1J axMWYvjNSEMvR2Bwco+Mzpwp1KbriwWjBZ/QB5g= X-Received: by 2002:a05:6a00:24d1:b0:63b:7a55:ae89 with SMTP id d17-20020a056a0024d100b0063b7a55ae89mr19497198pfv.27.1684849677518; Tue, 23 May 2023 06:47:57 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 26/27] qemu/atomic128: Add runtime test for FEAT_LSE2 Date: Tue, 23 May 2023 06:47:32 -0700 Message-Id: <20230523134733.678646-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With FEAT_LSE2, load and store of int128 is directly supported. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/aarch64/host/atomic128-ldst.h | 53 ++++++++++++++++------ 1 file changed, 40 insertions(+), 13 deletions(-) diff --git a/host/include/aarch64/host/atomic128-ldst.h b/host/include/aarch64/host/atomic128-ldst.h index 4b1360de39..5aabd6b978 100644 --- a/host/include/aarch64/host/atomic128-ldst.h +++ b/host/include/aarch64/host/atomic128-ldst.h @@ -11,27 +11,48 @@ #ifndef AARCH64_ATOMIC128_LDST_H #define AARCH64_ATOMIC128_LDST_H +#include "host/cpuinfo.h" +#include "tcg/debug-assert.h" + /* * Through gcc 10, aarch64 has no support for 128-bit atomics. * Through clang 16, without -march=armv8.4-a, __atomic_load_16 * is incorrectly expanded to a read-write operation. + * + * Anyway, this method allows runtime detection of FEAT_LSE2. */ -#define HAVE_ATOMIC128_RO 0 +#define HAVE_ATOMIC128_RO (cpuinfo & CPUINFO_LSE2) #define HAVE_ATOMIC128_RW 1 -Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); +static inline Int128 atomic16_read_ro(const Int128 *ptr) +{ + uint64_t l, h; + + tcg_debug_assert(HAVE_ATOMIC128_RO); + /* With FEAT_LSE2, 16-byte aligned LDP is atomic. */ + asm("ldp %[l], %[h], %[mem]" + : [l] "=r"(l), [h] "=r"(h) : [mem] "m"(*ptr)); + + return int128_make128(l, h); +} static inline Int128 atomic16_read_rw(Int128 *ptr) { uint64_t l, h; uint32_t tmp; - /* The load must be paired with the store to guarantee not tearing. */ - asm("0: ldxp %[l], %[h], %[mem]\n\t" - "stxp %w[tmp], %[l], %[h], %[mem]\n\t" - "cbnz %w[tmp], 0b" - : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + if (cpuinfo & CPUINFO_LSE2) { + /* With FEAT_LSE2, 16-byte aligned LDP is atomic. */ + asm("ldp %[l], %[h], %[mem]" + : [l] "=r"(l), [h] "=r"(h) : [mem] "m"(*ptr)); + } else { + /* The load must be paired with the store to guarantee not tearing. */ + asm("0: ldxp %[l], %[h], %[mem]\n\t" + "stxp %w[tmp], %[l], %[h], %[mem]\n\t" + "cbnz %w[tmp], 0b" + : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + } return int128_make128(l, h); } @@ -41,12 +62,18 @@ static inline void atomic16_set(Int128 *ptr, Int128 val) uint64_t l = int128_getlo(val), h = int128_gethi(val); uint64_t t1, t2; - /* Load into temporaries to acquire the exclusive access lock. */ - asm("0: ldxp %[t1], %[t2], %[mem]\n\t" - "stxp %w[t1], %[l], %[h], %[mem]\n\t" - "cbnz %w[t1], 0b" - : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) - : [l] "r"(l), [h] "r"(h)); + if (cpuinfo & CPUINFO_LSE2) { + /* With FEAT_LSE2, 16-byte aligned STP is atomic. */ + asm("stp %[l], %[h], %[mem]" + : [mem] "=m"(*ptr) : [l] "r"(l), [h] "r"(h)); + } else { + /* Load into temporaries to acquire the exclusive access lock. */ + asm("0: ldxp %[t1], %[t2], %[mem]\n\t" + "stxp %w[t1], %[l], %[h], %[mem]\n\t" + "cbnz %w[t1], 0b" + : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) + : [l] "r"(l), [h] "r"(h)); + } } #endif /* AARCH64_ATOMIC128_LDST_H */ From patchwork Tue May 23 13:47:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684930 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp1729792wrt; Tue, 23 May 2023 06:50:54 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7HDow2o0xEOGCbNUnk1uL/SxQHF/WHEoiuP5TdT9KAsx8F7oN3Us8Obz8OQC2S6BNIFGVs X-Received: by 2002:ad4:5942:0:b0:623:86a9:7696 with SMTP id eo2-20020ad45942000000b0062386a97696mr19213565qvb.5.1684849854506; Tue, 23 May 2023 06:50:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684849854; cv=none; d=google.com; s=arc-20160816; b=oS/4jMII0GLJ9zl3esdh7JTxeGQEuS5yC6K60VguauJ35OsEu0PZB8Z5/lVdk4+MXW MKE+wf3x50vVKhrls4GYhhxldi0/hfwzd5lvfua1wQe9P6MickdDnQcbMsPvIdYO1utc hpvgept1Ffsqd5rdvFWN+KRUxuA1RiffKVikDOXByS10skCPQOFSCmNtF5uurTPw5Flh LQn01zgtwFhGTlnhh8JEO9IcTFvq9S2ApVxXfHYI4CsF1kAwgZ0HaJRWsISDajSTonze E1a6p5eHA3tDUw+rLRqQNKmrgTKIUUsPCYX3ZRmnZ+DgoDeZfx0EjiGSZnS5tb77ez36 rocA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=FvkUN9jgmDAOLN4anBgxZjSYCqoddaXAUfRdd1Denco=; b=cQtnUErbw7eSUdBTEInoQaDG+nQnRiRQTASDJYiJi3x4+5+r4NsrEx7wjK/MQFeK8W 480o2RJI1+1EY/8IHZqz3QRKfM/dl3XUcgpDdKw91iDJqqTMHYlFV5abLNqpsUMb8r6w rzO0uBK44rcx5sSIeQRfT7AC+GkmvitoS+LdPnkmR9NWVRXN0FIl/7kc05tSiiTgPjGh ucl0OlErOmwuR5z9N3MUN5203s7yFBJn2D+PU1FWu/eBLpjap/cBH0R7hunSb4YLXbxF URntVTB7NPSy4engvGr9iiCzRfPkwImZNdB/vxaP0P3VeC4BB+K4lfhGbH/NAwwXx5Sd rdQw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Ukqy8hdL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q11-20020a056214194b00b00623851987c0si5195648qvk.150.2023.05.23.06.50.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 May 2023 06:50:54 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Ukqy8hdL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q1SMu-0001Ac-Bv; Tue, 23 May 2023 09:48:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q1SMs-00017r-4Z for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:02 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q1SMq-0001AT-If for qemu-devel@nongnu.org; Tue, 23 May 2023 09:48:01 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-64d426e63baso4400909b3a.0 for ; Tue, 23 May 2023 06:48:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684849678; x=1687441678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FvkUN9jgmDAOLN4anBgxZjSYCqoddaXAUfRdd1Denco=; b=Ukqy8hdLZBFZB8fCERx5VXrZEUf/5gIXs3k0aJmXOk6ORfQc+4CFnvp1F3DVahsVuY YUAl4tJY0Xyz8f47LzmoHvlI5psGIIT4Yz2yjup+7gOj4X2fj6CG4CtkL2zeU6invWCz LzJjPl9kPLy8KdN6mOQzpy+3Bt/LeEua2uHRS5Myqcm2cCJd0+MPK1wu6NnjECfmvjeI z0vvoNxt10i4M3fkOaimQkP14MlUMEg3JZ+Ajlm3y3EZUsAnQJP5K6E9I6yu8WuJWWlR owcPIOqBqyy023UI8reEAU7YZ4Xj08InH5T+Iz9rQoLWVReA+q4vqdrbNpYxNEtWC7x3 lqtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684849678; x=1687441678; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FvkUN9jgmDAOLN4anBgxZjSYCqoddaXAUfRdd1Denco=; b=SAr4g+a0GbMTJCXA7QKBOPqKke9yi+zKzktNIwjQ9kn35sTO2XFbmZdp+lKaAHiRi3 ljUC2XrP7aQeYq+yOJOO5Qkybb1tHBXT4mGu+/fU6kZC7kItIdZ/10m6OsSV6NL+4ELU X7dImBU2A8D9vRp+wE89p2RaQUAMJFKgez0Ka47vU7l9rmsr37IejPX0uVU29NLW13Wv uGVygvtpHDwEMM9h6j9tK7V+HRZ2JrvgWpxoPXM5Dsp7Q94eNcnvvsq21ceY8QwiXOLM 74fO/Son0gTx/coQQOnwHCiowBgm6rqWhPgFZarPUstqIKFmGLpXgwSpGEscXbqOjz46 DYbg== X-Gm-Message-State: AC+VfDwHFZUJmJSwISRzXUPPJ5lu5k4jULApk/ef5dbOquVDYZHObU48 KeZZu3HBjtMPmgUQxBoh74u2fLEjIJ8mJhK1vnM= X-Received: by 2002:a05:6a00:1788:b0:636:e52f:631e with SMTP id s8-20020a056a00178800b00636e52f631emr19931490pfg.1.1684849678344; Tue, 23 May 2023 06:47:58 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:c13a:d73:4f88:3654]) by smtp.gmail.com with ESMTPSA id j4-20020aa79284000000b0064d1349dc31sm5737122pfa.199.2023.05.23.06.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 May 2023 06:47:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: peter.maydell@linaro.org Subject: [PATCH v2 27/27] qemu/atomic128: Add x86_64 atomic128-ldst.h Date: Tue, 23 May 2023 06:47:33 -0700 Message-Id: <20230523134733.678646-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230523134733.678646-1-richard.henderson@linaro.org> References: <20230523134733.678646-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With CPUINFO_ATOMIC_VMOVDQA, we can perform proper atomic load/store without cmpxchg16b. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/x86_64/host/atomic128-ldst.h | 54 +++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 host/include/x86_64/host/atomic128-ldst.h diff --git a/host/include/x86_64/host/atomic128-ldst.h b/host/include/x86_64/host/atomic128-ldst.h new file mode 100644 index 0000000000..4be9071d3f --- /dev/null +++ b/host/include/x86_64/host/atomic128-ldst.h @@ -0,0 +1,54 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, x86_64 version. + * + * Copyright (C) 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_LDST_H +#define AARCH64_ATOMIC128_LDST_H + +#include "host/cpuinfo.h" +#include "tcg/debug-assert.h" + +#define HAVE_ATOMIC128_RO likely(cpuinfo & CPUINFO_ATOMIC_VMOVDQA) +#define HAVE_ATOMIC128_RW 1 + +static inline Int128 atomic16_read_ro(const Int128 *ptr) +{ + Int128Alias r; + + tcg_debug_assert(HAVE_ATOMIC128_RO); + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr)); + + return r.s; +} + +static inline Int128 atomic16_read_rw(Int128 *ptr) +{ + Int128Alias r; + + if (HAVE_ATOMIC128_RO) { + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr)); + } else { + r.i = __sync_val_compare_and_swap_16(ptr, 0, 0); + } + return r.s; +} + +static inline void atomic16_set(Int128 *ptr, Int128Alias val) +{ + if (HAVE_ATOMIC128_RO) { + asm("vmovdqa %1, %0" : "=m"(*ptr) : "x" (val.i)); + } else { + Int128Alias old; + do { + old.s = *ptr; + } while (!__sync_bool_compare_and_swap_16(ptr, old.i, val.i)); + } +} + +#endif /* AARCH64_ATOMIC128_LDST_H */