From patchwork Wed Sep 18 18:02:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173992 Delivered-To: patch@linaro.org Received: by 2002:a92:7e96:0:0:0:0:0 with SMTP id q22csp2777113ill; Wed, 18 Sep 2019 11:03:55 -0700 (PDT) X-Google-Smtp-Source: APXvYqykXoKQ1VDiGSxSY7kTGWqXmx/t47lwfoi1hhEH5M2msjF7X0mL/M1LwcXC8fr7a8Hqs56v X-Received: by 2002:a17:906:4a06:: with SMTP id w6mr8076079eju.214.1568829835758; Wed, 18 Sep 2019 11:03:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568829835; cv=none; d=google.com; s=arc-20160816; b=PAEw8hvElfUJ9SrgBf0EDD2Kg9ef61Q48gKvBPDKzg2Aqz1ONPm4dIvilIFTu2rDHl vwWz3YX9XmSFWy0X1L2/xwWyhk3dR/YqZ6t6gYm+gUA8Dzwvh/62n9R5OIQM77sJaAv9 7KMVIiVX8k9iX0rj7Avub37TxEUOKTAgQ7LfdKcdMSNra1p5N+Iah2XioZWQiG5mLaRM hqQXaEZgSdh8k3o4wYqLUeR++uiCxt7kStAIHBdj0b78ePo38yTNms58LeiwiP/6Z9Rv 9LyEsCpENSg6wjNBmhqcMZcMdltqjWKIVqwMW9pJiURMUKczahCamrxp/G+ANBwTvbrL XQGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=YucIarDiughcNjItqw4GzM0WuzRRGrmYnyWJbXHK1Xc=; b=PgjmapPq35GBpokEUSa+XhQWhsY6wu1zZz9RRApem89Z9EiVxdbqyn3yv7Pe/CTl// 7izJLT6JLbTM/ODyN2KLjgxwx0g5B5qQKuvKUJjNQazbDVjh0Lhz3QP7ETItg7J81OVB D5FNwrd51nFrNgVZqu7JE9agTZ2cTtji7eB2ZmK4HSk4CDIEeNTLfkJDbb6HqHrFvYV+ V5lAwl+0UZSd3Wnze8dn6lLdYY/wWAhDH8IAQD2lshAIn1YNVqxaX4f0ROx+Ht97ORTZ xJ6YsUwCS0czME5eW1A1edecnmZurhVx7FAw/KVkSZpGH5a8z3bBmL+J+U1IbM7/SWIm 34fA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=uFGlCSOW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g21si3919744edg.339.2019.09.18.11.03.55 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Sep 2019 11:03:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=uFGlCSOW; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33902 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIz-00020b-Dm for patch@linaro.org; Wed, 18 Sep 2019 14:03:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45835) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeI5-0001y5-P3 for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI4-0000RE-HV for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:57 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]:41085) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI4-0000QG-C4 for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:56 -0400 Received: by mail-pf1-x436.google.com with SMTP id q7so487625pfh.8 for ; Wed, 18 Sep 2019 11:02:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=YucIarDiughcNjItqw4GzM0WuzRRGrmYnyWJbXHK1Xc=; b=uFGlCSOWaQ9pQd3SvbCiO9C6FlvhlqZSoeFpvmgEgvp3iwl1XnrYNWgeu3zb5nUqd3 g2P5J2ghYZUwV2iS7KA3lfqAyOq3Vb/oDz7V7zb3BFWQ1du1w89Q73UzS8ZYRmrUJPdk VkAzEkZ9XcoWj0kmsHR+bNH3f9jxmA8jAftVbTBItI+2Ia+C9Qq6IYqrpxzHIfs7atiX 3n4c+MGZ6TWyd9BnYtbIkr5T1i1dfBvHjhWA8T43KTW8jP2AUKnmFFKi7aCRSNwh2/xq jt8xd9JMbz/WFG+97tLpYltfNdmfkVSARZqo1Or9dADhbKdSBM0BsbZ4uBm+f0nKQiVL tcow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=YucIarDiughcNjItqw4GzM0WuzRRGrmYnyWJbXHK1Xc=; b=dPZOaePNTQuy1AyR2mvzCJahFgOJFKAYCnWmzxpRTOoa9MyjQaH/XvYQI59A7TLHJE jGP9vNHK5U2OJ/BZuni+U5NQlVpvxZ0Gh7Q0CDQRf8TKn8rzKAaY5QXUeR3Paeus5I8W Ep4IalCshhwKlXIw56mWZq8pCUm3oC91JLvAcrkNIgSxmGTVZFgqObE1gcgkd3PMpyYI zze8Uzm1urnVRtO5XeXTs58bFKCnHNB1ANGWHX8tPRebr4+JWX5zzS+BCJmXweYHyXU+ GCjfBiRcimNkXmOqj1vKzOG63rTp6g95+ll+7vjaDJ4OWtiQBJB/I0XAjk/1LSFpH/xj GHNg== X-Gm-Message-State: APjAAAX8m3hGz2lPSo//HS2VvGPeZ+rda0yXKFwWPhXrvmaJTtO7dH9F n4blPaAG6i1r+ZaII59/HwKzx+yZcEI= X-Received: by 2002:a63:cf0a:: with SMTP id j10mr5032746pgg.388.1568829774491; Wed, 18 Sep 2019 11:02:54 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:47 -0700 Message-Id: <20190918180251.32003-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::436 Subject: [Qemu-devel] [RFC v2 1/5] cputlb: Disable __always_inline__ without optimization X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This forced inlining can result in missing symbols, which makes a debugging build harder to follow. Reported-by: Peter Maydell Signed-off-by: Richard Henderson --- include/qemu/compiler.h | 11 +++++++++++ accel/tcg/cputlb.c | 4 ++-- 2 files changed, 13 insertions(+), 2 deletions(-) -- 2.17.1 Reviewed-by: David Hildenbrand diff --git a/include/qemu/compiler.h b/include/qemu/compiler.h index 09fc44cca4..d6d400c523 100644 --- a/include/qemu/compiler.h +++ b/include/qemu/compiler.h @@ -170,6 +170,17 @@ # define QEMU_NONSTRING #endif +/* + * Forced inlining may be desired to encourage constant propagation + * of function parameters. However, it can also make debugging harder, + * so disable it for a non-optimizing build. + */ +#if defined(__OPTIMIZE__) && __has_attribute(always_inline) +#define QEMU_ALWAYS_INLINE __attribute__((always_inline)) +#else +#define QEMU_ALWAYS_INLINE +#endif + /* Implement C11 _Generic via GCC builtins. Example: * * QEMU_GENERIC(x, (float, sinf), (long double, sinl), sin) (x) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index abae79650c..2222b87764 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1281,7 +1281,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); -static inline uint64_t __attribute__((always_inline)) +static inline uint64_t QEMU_ALWAYS_INLINE load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, bool code_read, FullLoadHelper *full_load) @@ -1530,7 +1530,7 @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr, * Store Helpers */ -static inline void __attribute__((always_inline)) +static inline void QEMU_ALWAYS_INLINE store_helper(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op) { From patchwork Wed Sep 18 18:02:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173995 Delivered-To: patch@linaro.org Received: by 2002:ac9:19ad:0:0:0:0:0 with SMTP id d45csp15341oce; Wed, 18 Sep 2019 11:07:57 -0700 (PDT) X-Google-Smtp-Source: APXvYqyr7KfCJpcSObGjMeTSCYesbFfbg6rI1WpFmItsD/XL9bUdBpLVzZfbGXEnwFKi7qXFNvQc X-Received: by 2002:a0c:ad19:: with SMTP id u25mr4482933qvc.239.1568830077761; Wed, 18 Sep 2019 11:07:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568830077; cv=none; d=google.com; s=arc-20160816; b=DY6PseAhnIP0OnVZqrcvt6JV+fsIyCbH/nxn2o5TmSuwyxFbGTzaq3cKB8p8S3kqk9 Sz0S8//WIR24EV57E85ZM3a7HZ1Qv9FLtn2VaK5SXsPYrL4YFXmWV9w7bitaJPiHGc61 KuIbOCEfiI/QWxnUftggoy6E8mmp2aTJn0Vn5sHeDwVE1JNqIccqTKbQIe/k3nXIsZNc iVTzM5hVbJI6R149bUBoclazp5DO+mgEqj1iEmCh481yWIdpVx9VmeRB4sYNDyBwD9Gm 14QFEXBW+z/DTDbOcFIDw6UYSDjENjN5nzsaImGkgqg2D2Owhb/6TJIBEgKXG6KQmKLU naEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=3cfrqeC7LyJB/ROXN1SmwKtiOawGwvTqMSrbdq/eHoM=; b=D2Gw5cVPW0wt9yVg/jm63aS0FBNbsX6eoJur0z3FjuoyXfJ6YIJ8YwKgMQRccZ4h3I HDIDVYBI43tirFRl3PMHGxL8xf5zU7fJgKgJrBL+X4eJLCzE0PAWef+qwTY61gD8K994 3d33igM4FAd1LNR+qPIcXsAVAjljT1xEuNFSy9PWSHyxiQ48FufvzxIqsMQbBBrAoyiD HSEXU0MgOAndTH8lozeGTe5ExqcApqFjBAr4YiqeDlZ7LBoULphIVoT2y4N8et+1JCqt VeOX+51BDr+dXzxu+ar+/meSdHBX5xtWrR4XKcxO1lbufkUQ11NB7K8dRG9UHkZlShSa EjxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=rMDmqTEg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 198si4453249qkj.34.2019.09.18.11.07.57 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Sep 2019 11:07:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=rMDmqTEg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34000 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeMv-0005hP-69 for patch@linaro.org; Wed, 18 Sep 2019 14:07:57 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45852) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeI7-0001zg-Ox for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:02 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI5-0000Rn-Up for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:59 -0400 Received: from mail-pf1-x442.google.com ([2607:f8b0:4864:20::442]:45460) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI5-0000RO-Lk for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:57 -0400 Received: by mail-pf1-x442.google.com with SMTP id y72so470757pfb.12 for ; Wed, 18 Sep 2019 11:02:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3cfrqeC7LyJB/ROXN1SmwKtiOawGwvTqMSrbdq/eHoM=; b=rMDmqTEgtsk7zVSCslbphhy6+8evakesTNdLdxDMljXF9O24DEjuHvazZGHJLRMZSm Ss3pD5Sz3T2a9vHlpT/zytApIz1+f5/qnJY8CUVLBgsBV23oEetbrMmF1gp4mlkFpOiL pRCuN4NrTxYEu54HQrJp5CzUOzNiye+dxumMid5vUcw7PXDy3QPxyMaxlC459vU5xZMK YK/e1l8X1vGM8TD/hY/kzWEFaPbjf3Y52R0j1H9u7ZYb/p185nhNZljuVCUJim2YkxMO z9mVxTtvBq6AH+7ZlNlRcsjJ1Zn7qLZs0e2Ez88n/N8gXVjxabzJZIfCpyJMpz4Jy8VY T42A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3cfrqeC7LyJB/ROXN1SmwKtiOawGwvTqMSrbdq/eHoM=; b=HpMIc7LiLSZDDw7FqHBiht1X/oDtQZPyPH78KLjHe7mY0GgPdDRIwTS+sVaUFuHw02 yHCsEhRvQ5WiUBQE0T4ch4tVuN4+a5EkKmbt0H8wfyHvLTLZPCqRWLQ+zr2Ml2eXn6b0 CZR7u3ZtDP1c6Y8DSzRCgC9Ho48EykuF1TCmRTpyidRbawXxv2HXHySSmrqgO03ZGC7Z 0E9xK90RIyNMARJFvEq4xcD2C9wBWG1h5Ppj8cGuFRHyNjcZAnipR3D2rEtAVPFCfm6V R2W6gsxDEj4d5+Yvy9UVQ/ObjGk8o+ycu9j8suuCchqNlR0XiBcvl372++spf726fln+ rRfg== X-Gm-Message-State: APjAAAXIGnQ+ZShuAAIyAn4ab1ndpcnzhQOfuq1tX2B0l1UIKktS/mq4 ndZiQ01QDsthzPN3g257S04LesYKS08= X-Received: by 2002:a63:2b0c:: with SMTP id r12mr5159576pgr.206.1568829776074; Wed, 18 Sep 2019 11:02:56 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:48 -0700 Message-Id: <20190918180251.32003-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::442 Subject: [Qemu-devel] [RFC v2 2/5] cputlb: Replace switches in load/store_helper with callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Add a function parameter to perform the actual load/store to ram. With optimization, this results in identical code. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 159 +++++++++++++++++++++++---------------------- 1 file changed, 83 insertions(+), 76 deletions(-) -- 2.17.1 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 2222b87764..b4a63d3928 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1280,11 +1280,38 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, typedef uint64_t FullLoadHelper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr); +typedef uint64_t LoadHelper(const void *); + +/* Wrap the unaligned load helpers to that they have a common signature. */ +static inline uint64_t wrap_ldub(const void *haddr) +{ + return ldub_p(haddr); +} + +static inline uint64_t wrap_lduw_be(const void *haddr) +{ + return lduw_be_p(haddr); +} + +static inline uint64_t wrap_lduw_le(const void *haddr) +{ + return lduw_le_p(haddr); +} + +static inline uint64_t wrap_ldul_be(const void *haddr) +{ + return (uint32_t)ldl_be_p(haddr); +} + +static inline uint64_t wrap_ldul_le(const void *haddr) +{ + return (uint32_t)ldl_le_p(haddr); +} static inline uint64_t QEMU_ALWAYS_INLINE load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, bool code_read, - FullLoadHelper *full_load) + FullLoadHelper *full_load, LoadHelper *direct) { uintptr_t mmu_idx = get_mmuidx(oi); uintptr_t index = tlb_index(env, mmu_idx, addr); @@ -1373,33 +1400,7 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, do_aligned_access: haddr = (void *)((uintptr_t)addr + entry->addend); - switch (op) { - case MO_UB: - res = ldub_p(haddr); - break; - case MO_BEUW: - res = lduw_be_p(haddr); - break; - case MO_LEUW: - res = lduw_le_p(haddr); - break; - case MO_BEUL: - res = (uint32_t)ldl_be_p(haddr); - break; - case MO_LEUL: - res = (uint32_t)ldl_le_p(haddr); - break; - case MO_BEQ: - res = ldq_be_p(haddr); - break; - case MO_LEQ: - res = ldq_le_p(haddr); - break; - default: - g_assert_not_reached(); - } - - return res; + return direct(haddr); } /* @@ -1415,7 +1416,8 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, static uint64_t full_ldub_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { - return load_helper(env, addr, oi, retaddr, MO_UB, false, full_ldub_mmu); + return load_helper(env, addr, oi, retaddr, MO_UB, false, + full_ldub_mmu, wrap_ldub); } tcg_target_ulong helper_ret_ldub_mmu(CPUArchState *env, target_ulong addr, @@ -1428,7 +1430,7 @@ static uint64_t full_le_lduw_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, false, - full_le_lduw_mmu); + full_le_lduw_mmu, wrap_lduw_le); } tcg_target_ulong helper_le_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1441,7 +1443,7 @@ static uint64_t full_be_lduw_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, false, - full_be_lduw_mmu); + full_be_lduw_mmu, wrap_lduw_be); } tcg_target_ulong helper_be_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1454,7 +1456,7 @@ static uint64_t full_le_ldul_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, false, - full_le_ldul_mmu); + full_le_ldul_mmu, wrap_ldul_le); } tcg_target_ulong helper_le_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1467,7 +1469,7 @@ static uint64_t full_be_ldul_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, false, - full_be_ldul_mmu); + full_be_ldul_mmu, wrap_ldul_be); } tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1480,14 +1482,14 @@ uint64_t helper_le_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, false, - helper_le_ldq_mmu); + helper_le_ldq_mmu, ldq_le_p); } uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, false, - helper_be_ldq_mmu); + helper_be_ldq_mmu, ldq_be_p); } /* @@ -1530,9 +1532,38 @@ tcg_target_ulong helper_be_ldsl_mmu(CPUArchState *env, target_ulong addr, * Store Helpers */ +typedef void StoreHelper(void *, uint64_t); + +/* Wrap the unaligned store helpers to that they have a common signature. */ +static inline void wrap_stb(void *haddr, uint64_t val) +{ + stb_p(haddr, val); +} + +static inline void wrap_stw_be(void *haddr, uint64_t val) +{ + stw_be_p(haddr, val); +} + +static inline void wrap_stw_le(void *haddr, uint64_t val) +{ + stw_le_p(haddr, val); +} + +static inline void wrap_stl_be(void *haddr, uint64_t val) +{ + stl_be_p(haddr, val); +} + +static inline void wrap_stl_le(void *haddr, uint64_t val) +{ + stl_le_p(haddr, val); +} + static inline void QEMU_ALWAYS_INLINE store_helper(CPUArchState *env, target_ulong addr, uint64_t val, - TCGMemOpIdx oi, uintptr_t retaddr, MemOp op) + TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, + StoreHelper *direct) { uintptr_t mmu_idx = get_mmuidx(oi); uintptr_t index = tlb_index(env, mmu_idx, addr); @@ -1657,74 +1688,49 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, do_aligned_access: haddr = (void *)((uintptr_t)addr + entry->addend); - switch (op) { - case MO_UB: - stb_p(haddr, val); - break; - case MO_BEUW: - stw_be_p(haddr, val); - break; - case MO_LEUW: - stw_le_p(haddr, val); - break; - case MO_BEUL: - stl_be_p(haddr, val); - break; - case MO_LEUL: - stl_le_p(haddr, val); - break; - case MO_BEQ: - stq_be_p(haddr, val); - break; - case MO_LEQ: - stq_le_p(haddr, val); - break; - default: - g_assert_not_reached(); - break; - } + direct(haddr, val); } void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_UB); + store_helper(env, addr, val, oi, retaddr, MO_UB, wrap_stb); } void helper_le_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUW); + store_helper(env, addr, val, oi, retaddr, MO_LEUW, wrap_stw_le); } void helper_be_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUW); + store_helper(env, addr, val, oi, retaddr, MO_BEUW, wrap_stw_be); } void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUL); + store_helper(env, addr, val, oi, retaddr, MO_LEUL, wrap_stl_le); } void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUL); + store_helper(env, addr, val, oi, retaddr, MO_BEUL, wrap_stl_be); } void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEQ); + store_helper(env, addr, val, oi, retaddr, MO_LEQ, stq_le_p); } void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEQ); + store_helper(env, addr, val, oi, retaddr, MO_BEQ, stq_be_p); } /* First set of helpers allows passing in of OI and RETADDR. This makes @@ -1789,7 +1795,8 @@ void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, static uint64_t full_ldub_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { - return load_helper(env, addr, oi, retaddr, MO_8, true, full_ldub_cmmu); + return load_helper(env, addr, oi, retaddr, MO_8, true, + full_ldub_cmmu, wrap_ldub); } uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr, @@ -1802,7 +1809,7 @@ static uint64_t full_le_lduw_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, true, - full_le_lduw_cmmu); + full_le_lduw_cmmu, wrap_lduw_le); } uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1815,7 +1822,7 @@ static uint64_t full_be_lduw_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, true, - full_be_lduw_cmmu); + full_be_lduw_cmmu, wrap_lduw_be); } uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1828,7 +1835,7 @@ static uint64_t full_le_ldul_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, true, - full_le_ldul_cmmu); + full_le_ldul_cmmu, wrap_ldul_le); } uint32_t helper_le_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1841,7 +1848,7 @@ static uint64_t full_be_ldul_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, true, - full_be_ldul_cmmu); + full_be_ldul_cmmu, wrap_ldul_be); } uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1854,12 +1861,12 @@ uint64_t helper_le_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, true, - helper_le_ldq_cmmu); + helper_le_ldq_cmmu, ldq_le_p); } uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, true, - helper_be_ldq_cmmu); + helper_be_ldq_cmmu, ldq_be_p); } From patchwork Wed Sep 18 18:02:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173997 Delivered-To: patch@linaro.org Received: by 2002:ac9:19ad:0:0:0:0:0 with SMTP id d45csp19890oce; Wed, 18 Sep 2019 11:11:58 -0700 (PDT) X-Google-Smtp-Source: APXvYqwenKgLXqrg3kCNAWdRtOY7zrWgx6UZBiLk0LggeYcpFZgrXrXqwfhH8cWF/wdZoJAwVWlQ X-Received: by 2002:a50:e005:: with SMTP id e5mr11782516edl.279.1568830318449; Wed, 18 Sep 2019 11:11:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568830318; cv=none; d=google.com; s=arc-20160816; b=u/jpNgEIvDjZOokI7DFC28NqrOfjPIC9GID7VOfS/L5iXxHYd53kqtzziMMRm6VVwm wq4G9E3Hk2cghA2CDjwy2AUjtFNL4rYik8bUdBHm2b81acJ6akwJ8mu7n1t02E/u7rAU +38c4oebZ93BX2WQRMldiFHUN5AyJphXhZM+5ePJ9qYHc73tIPVgYvBF/pbcUZ/7RYTL MKH0C70WlqeP7e0foXBFscWdyxA5s/NHnQpXskTKTfYblMBcfa4f4P9/jpo0KfBLkCft eA0uVCxc3VqhI2GKFVOTDsE4K3W7lX0z5HGzBMLeTBAJ86F0D6cM2k91uXP5HmDBjv6l /gyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=6ZfbVwaLbCiGgpEw6rDTymD03FTM4V/YWYRYGaFUxL8=; b=vZd66AsYRkyb7L4hs6b6gykBkLS34JOlMgM5mkFd0unVVO4izTI48hAivPYS/Z6lwE zE/V9HQFV5ICT/KKhamBTekFNHzwctoskUIXlN8BWFwRqZFxWkkaxNBanCfatFbhlvE/ 5lDJCYgLUB8ud6abTNb6+HZ6beHAV3BpNBIW2x1mJivza9Z1xthbkxtwJ6zpxPMntbIb LDjEH8j0T5Sb1sHoY8UNm/hHMC0v8tG92Sq/TP7jIPQjv2WX37Z2kCjLCNtjjG02Awrt 0iS+tZHDAS7ZSMZQRK9Sohqtywya3teSnTNt1Yguogec+jxlJ5lUSOksB1c7OWGKvfjz sQAw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=xW6Pxapb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d3si3174216eja.30.2019.09.18.11.11.58 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Sep 2019 11:11:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=xW6Pxapb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34040 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeQm-0000gn-RX for patch@linaro.org; Wed, 18 Sep 2019 14:11:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45881) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIC-00022R-MN for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI7-0000SF-5T for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:02 -0400 Received: from mail-pg1-x542.google.com ([2607:f8b0:4864:20::542]:41358) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI6-0000S5-Ts for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:02:59 -0400 Received: by mail-pg1-x542.google.com with SMTP id x15so283375pgg.8 for ; Wed, 18 Sep 2019 11:02:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=6ZfbVwaLbCiGgpEw6rDTymD03FTM4V/YWYRYGaFUxL8=; b=xW6Pxapb4zhFSPriqaOFq5+JoB/3MzVaCDoku904vsV6YBnLN4Qv0r459FXGfSz74z +sj6VkGcrFNoipwbiVCwIeJiqrrpX/hd+bV8Mg0pLT5m6/s919OqRaItLUK4tFyqEIzj Q4lCorWDxtOsgikaC59FpsOD8lVAkgHirHReT3r8Otv21nQ8HIffZSqrUeNdX81/Gjsf +frDnS3OpQGYxoX3StqVLEDBa9C1aVBE6xLf9V92EP3VMXEjBnX4D2sTDUkDmQh0yjdf Go+AlUUBxseBbg+1c4bUBi/9xrU+Y4lNEhkr4AxksEeC042tqmEWn/8DA6FdFyAN4jKc TGQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=6ZfbVwaLbCiGgpEw6rDTymD03FTM4V/YWYRYGaFUxL8=; b=T4VkzmvitVaEd+eUCX3MqUSNQ7IwWhgtzLwi2dV6P94yJ9f+SHXJrBry+C8a6kyHIq sY/1TAai8Al0ECSA+aCQaMaS9cBSAJDZv9Xq1WNfSNPsIBK7bqxecNdVcDO7BsAeVFmY 9PO35F6NfhiBwePXuQdWi4H5YgjM6+AdFcEnnLCH1Lzr1/FaB9FhbB9KJLkFJ16y6VNv ad24yZAURPzwyvkChxxanmQvmFMhqOG5S6tlNMmJdoIl3dgaeLdmDcENFXAOMxOStXoB FKtJnhMYZhSzhOuQlpIjR9yg/g8zBSK8VxNs4pZ0YdPiW6E562pw3JtkDveTbQPx/RVl lXKw== X-Gm-Message-State: APjAAAVCsebKkPb4ioSSbk4ZBtBhq6K+KOpZFM17hzv8A0Jna/VnGJ3/ VXyJnzbEPEkmxEXYi7SRzBrJM9d4My8= X-Received: by 2002:a63:eb52:: with SMTP id b18mr2716710pgk.71.1568829777336; Wed, 18 Sep 2019 11:02:57 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:49 -0700 Message-Id: <20190918180251.32003-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::542 Subject: [Qemu-devel] [RFC v2 3/5] cputlb: Introduce TLB_BSWAP X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Handle bswap on ram directly in load/store_helper. This fixes a bug with the previous implementation in that one cannot use the I/O path for RAM. Fixes: a26fc6f5152b47f1 Signed-off-by: Richard Henderson --- include/exec/cpu-all.h | 2 + accel/tcg/cputlb.c | 105 +++++++++++++++++++++-------------------- 2 files changed, 57 insertions(+), 50 deletions(-) -- 2.17.1 diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h index d2d443c4f9..3928edab9a 100644 --- a/include/exec/cpu-all.h +++ b/include/exec/cpu-all.h @@ -331,6 +331,8 @@ CPUArchState *cpu_copy(CPUArchState *env); #define TLB_MMIO (1 << (TARGET_PAGE_BITS - 3)) /* Set if TLB entry contains a watchpoint. */ #define TLB_WATCHPOINT (1 << (TARGET_PAGE_BITS - 4)) +/* Set if TLB entry requires byte swap. */ +#define TLB_BSWAP (1 << (TARGET_PAGE_BITS - 5)) /* Use this mask to check interception with an alignment mask * in a TCG backend. diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b4a63d3928..7301f9e699 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -737,8 +737,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, address |= TLB_INVALID_MASK; } if (attrs.byte_swap) { - /* Force the access through the I/O slow path. */ - address |= TLB_MMIO; + address |= TLB_BSWAP; } if (!memory_region_is_ram(section->mr) && !memory_region_is_romd(section->mr)) { @@ -901,10 +900,6 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry, bool locked = false; MemTxResult r; - if (iotlbentry->attrs.byte_swap) { - op ^= MO_BSWAP; - } - section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr = section->mr; mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; @@ -947,10 +942,6 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry, bool locked = false; MemTxResult r; - if (iotlbentry->attrs.byte_swap) { - op ^= MO_BSWAP; - } - section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr = section->mr; mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; @@ -1311,7 +1302,8 @@ static inline uint64_t wrap_ldul_le(const void *haddr) static inline uint64_t QEMU_ALWAYS_INLINE load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, bool code_read, - FullLoadHelper *full_load, LoadHelper *direct) + FullLoadHelper *full_load, LoadHelper *direct, + LoadHelper *direct_swap) { uintptr_t mmu_idx = get_mmuidx(oi); uintptr_t index = tlb_index(env, mmu_idx, addr); @@ -1361,17 +1353,22 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, /* On watchpoint hit, this will longjmp out. */ cpu_check_watchpoint(env_cpu(env), addr, size, iotlbentry->attrs, BP_MEM_READ, retaddr); - - /* The backing page may or may not require I/O. */ - tlb_addr &= ~TLB_WATCHPOINT; - if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) { - goto do_aligned_access; - } } /* Handle I/O access. */ - return io_readx(env, iotlbentry, mmu_idx, addr, - retaddr, access_type, op); + if (likely(tlb_addr & TLB_MMIO)) { + return io_readx(env, iotlbentry, mmu_idx, addr, + retaddr, access_type, + op ^ (tlb_addr & TLB_BSWAP ? MO_BSWAP : 0)); + } + + haddr = (void *)((uintptr_t)addr + entry->addend); + + if (unlikely(tlb_addr & TLB_BSWAP)) { + return direct_swap(haddr); + } else { + return direct(haddr); + } } /* Handle slow unaligned access (it spans two pages or IO). */ @@ -1398,7 +1395,6 @@ load_helper(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, return res & MAKE_64BIT_MASK(0, size * 8); } - do_aligned_access: haddr = (void *)((uintptr_t)addr + entry->addend); return direct(haddr); } @@ -1417,7 +1413,7 @@ static uint64_t full_ldub_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_UB, false, - full_ldub_mmu, wrap_ldub); + full_ldub_mmu, wrap_ldub, wrap_ldub); } tcg_target_ulong helper_ret_ldub_mmu(CPUArchState *env, target_ulong addr, @@ -1430,7 +1426,7 @@ static uint64_t full_le_lduw_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, false, - full_le_lduw_mmu, wrap_lduw_le); + full_le_lduw_mmu, wrap_lduw_le, wrap_lduw_be); } tcg_target_ulong helper_le_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1443,7 +1439,7 @@ static uint64_t full_be_lduw_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, false, - full_be_lduw_mmu, wrap_lduw_be); + full_be_lduw_mmu, wrap_lduw_be, wrap_lduw_le); } tcg_target_ulong helper_be_lduw_mmu(CPUArchState *env, target_ulong addr, @@ -1456,7 +1452,7 @@ static uint64_t full_le_ldul_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, false, - full_le_ldul_mmu, wrap_ldul_le); + full_le_ldul_mmu, wrap_ldul_le, wrap_ldul_be); } tcg_target_ulong helper_le_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1469,7 +1465,7 @@ static uint64_t full_be_ldul_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, false, - full_be_ldul_mmu, wrap_ldul_be); + full_be_ldul_mmu, wrap_ldul_be, wrap_ldul_le); } tcg_target_ulong helper_be_ldul_mmu(CPUArchState *env, target_ulong addr, @@ -1482,14 +1478,14 @@ uint64_t helper_le_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, false, - helper_le_ldq_mmu, ldq_le_p); + helper_le_ldq_mmu, ldq_le_p, ldq_be_p); } uint64_t helper_be_ldq_mmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, false, - helper_be_ldq_mmu, ldq_be_p); + helper_be_ldq_mmu, ldq_be_p, ldq_le_p); } /* @@ -1563,7 +1559,7 @@ static inline void wrap_stl_le(void *haddr, uint64_t val) static inline void QEMU_ALWAYS_INLINE store_helper(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr, MemOp op, - StoreHelper *direct) + StoreHelper *direct, StoreHelper *direct_swap) { uintptr_t mmu_idx = get_mmuidx(oi); uintptr_t index = tlb_index(env, mmu_idx, addr); @@ -1608,16 +1604,22 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, /* On watchpoint hit, this will longjmp out. */ cpu_check_watchpoint(env_cpu(env), addr, size, iotlbentry->attrs, BP_MEM_WRITE, retaddr); - - /* The backing page may or may not require I/O. */ - tlb_addr &= ~TLB_WATCHPOINT; - if ((tlb_addr & ~TARGET_PAGE_MASK) == 0) { - goto do_aligned_access; - } } /* Handle I/O access. */ - io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op); + if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) { + io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, + op ^ (tlb_addr & TLB_BSWAP ? MO_BSWAP : 0)); + return; + } + + haddr = (void *)((uintptr_t)addr + entry->addend); + + if (unlikely(tlb_addr & TLB_BSWAP)) { + direct_swap(haddr, val); + } else { + direct(haddr, val); + } return; } @@ -1686,7 +1688,6 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, return; } - do_aligned_access: haddr = (void *)((uintptr_t)addr + entry->addend); direct(haddr, val); } @@ -1694,43 +1695,47 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, void helper_ret_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_UB, wrap_stb); + store_helper(env, addr, val, oi, retaddr, MO_UB, wrap_stb, wrap_stb); } void helper_le_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUW, wrap_stw_le); + store_helper(env, addr, val, oi, retaddr, MO_LEUW, + wrap_stw_le, wrap_stw_be); } void helper_be_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUW, wrap_stw_be); + store_helper(env, addr, val, oi, retaddr, MO_BEUW, + wrap_stw_be, wrap_stw_le); } void helper_le_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEUL, wrap_stl_le); + store_helper(env, addr, val, oi, retaddr, MO_LEUL, + wrap_stl_le, wrap_stl_be); } void helper_be_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEUL, wrap_stl_be); + store_helper(env, addr, val, oi, retaddr, MO_BEUL, + wrap_stl_be, wrap_stl_le); } void helper_le_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_LEQ, stq_le_p); + store_helper(env, addr, val, oi, retaddr, MO_LEQ, stq_le_p, stq_be_p); } void helper_be_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, TCGMemOpIdx oi, uintptr_t retaddr) { - store_helper(env, addr, val, oi, retaddr, MO_BEQ, stq_be_p); + store_helper(env, addr, val, oi, retaddr, MO_BEQ, stq_be_p, stq_le_p); } /* First set of helpers allows passing in of OI and RETADDR. This makes @@ -1796,7 +1801,7 @@ static uint64_t full_ldub_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_8, true, - full_ldub_cmmu, wrap_ldub); + full_ldub_cmmu, wrap_ldub, wrap_ldub); } uint8_t helper_ret_ldb_cmmu(CPUArchState *env, target_ulong addr, @@ -1809,7 +1814,7 @@ static uint64_t full_le_lduw_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUW, true, - full_le_lduw_cmmu, wrap_lduw_le); + full_le_lduw_cmmu, wrap_lduw_le, wrap_lduw_be); } uint16_t helper_le_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1822,7 +1827,7 @@ static uint64_t full_be_lduw_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUW, true, - full_be_lduw_cmmu, wrap_lduw_be); + full_be_lduw_cmmu, wrap_lduw_be, wrap_lduw_le); } uint16_t helper_be_ldw_cmmu(CPUArchState *env, target_ulong addr, @@ -1835,7 +1840,7 @@ static uint64_t full_le_ldul_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEUL, true, - full_le_ldul_cmmu, wrap_ldul_le); + full_le_ldul_cmmu, wrap_ldul_le, wrap_ldul_be); } uint32_t helper_le_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1848,7 +1853,7 @@ static uint64_t full_be_ldul_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEUL, true, - full_be_ldul_cmmu, wrap_ldul_be); + full_be_ldul_cmmu, wrap_ldul_be, wrap_ldul_le); } uint32_t helper_be_ldl_cmmu(CPUArchState *env, target_ulong addr, @@ -1861,12 +1866,12 @@ uint64_t helper_le_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_LEQ, true, - helper_le_ldq_cmmu, ldq_le_p); + helper_le_ldq_cmmu, ldq_le_p, ldq_be_p); } uint64_t helper_be_ldq_cmmu(CPUArchState *env, target_ulong addr, TCGMemOpIdx oi, uintptr_t retaddr) { return load_helper(env, addr, oi, retaddr, MO_BEQ, true, - helper_be_ldq_cmmu, ldq_be_p); + helper_be_ldq_cmmu, ldq_be_p, ldq_le_p); } From patchwork Wed Sep 18 18:02:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173993 Delivered-To: patch@linaro.org Received: by 2002:ac9:19ad:0:0:0:0:0 with SMTP id d45csp14862oce; Wed, 18 Sep 2019 11:07:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqwHeA/OLiXeLv65J+bBNFSWK6K//UMfOQfTBfHOMH1qJ2VuE5B0Hy0bIbrxgaw6pfJZJQOb X-Received: by 2002:a17:906:4b47:: with SMTP id j7mr9307684ejv.63.1568830052028; Wed, 18 Sep 2019 11:07:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568830052; cv=none; d=google.com; s=arc-20160816; b=pUZbvffHp8T1FIbsHpgkcQvnWQvOL9iSxYzUnMzEIIALpRvj3iHHPnXilieHg5ltr2 vFSe8Py9VUIFsU2tEP5NjUsYblnE4uhUm1m2NCnZhsAl8Ga0OktIUCRPMgbBVEM0RNSV tindcFYIGTnHZzuTzHUOgB9HyDtY9C73Tg4lQZapYYnconqpXwfHApVMeCbzcjIW6FyV to+wN4aTsdjlWAMUqY6zUUnwzI8d0bM+7vygF2iFzO0ECrXNlLSc9aTWdHUlOlS/MJqu tS42giZQolLJTab8zdQafg1cM93JecFa66hKGrzW6Of+QceAI969KvMDFM6IbC+9Vkkz gBxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=0As7gB6KKU9NL64qrnl7+jEEiK3R3QPZ3EzF2mVrRe8=; b=mlQMuyix54CwPeGx2dWXzmXxJTDW3JThqLx2ldXegbHkCY9CivczWoZWFHxGCb4rPE Z2ajhdFdocmQfyNURDlPHvXuCQFubkRwRSXHyYTlIIaTe6JzOdX5qe5jV2SUR+40Ye95 lpDzAdMTu7SYJ1K+TLlWcg231WQbjIq/Za6M2oIL9mERiQL4RhaDfdy+tXHCok5ZvtxY 43pMzMtV+lejgWCGkezjFc7+5xrPvjTbL7qzkLiAWJuQAv/5quKu18bly90Q6uXG4M8J yu/vJCTOoNncqxXeds+BafCZbrSvzLIplEbeMEpQhfPo8xI1Ffz/e4qGPHndw4Mp7QQM C0uA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=AFAPrM0p; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z12si3307939eju.15.2019.09.18.11.07.31 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Sep 2019 11:07:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=AFAPrM0p; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33990 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeMU-0005HH-Iz for patch@linaro.org; Wed, 18 Sep 2019 14:07:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45886) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIC-00022S-Rs for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeI8-0000T1-I9 for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:02 -0400 Received: from mail-pl1-x641.google.com ([2607:f8b0:4864:20::641]:44257) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeI8-0000Sb-CL for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:00 -0400 Received: by mail-pl1-x641.google.com with SMTP id q15so303191pll.11 for ; Wed, 18 Sep 2019 11:03:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0As7gB6KKU9NL64qrnl7+jEEiK3R3QPZ3EzF2mVrRe8=; b=AFAPrM0pPH8ESex+gy/AqU0FMEmGTp3q4xlC75z7dg/HBbHpV+jHURa5omqxuNmQqs mRT/8AH+kApx+llQ37YUijo87FqkandZlwIRLwhzB02az24ybzFMvXlajlSNE5qeTnQO KD/p7ibsGV27JHWa1xOT6z55LMYKCQb9e3r3dOWFeF24FcTBDZ38fPVGGgdhVjWnVv2c 0vmXXANGYR0Ks/j8uimqYidQLmG0QYdcsjn8JNxiLa/M60jyPp/5c7Cvg9OF+KnLVjN0 jixp1qc7bDuvT6EtrzqSTZhPM4ojvSr7kBVIzH89Fdax4Sg5Srkc3crK1+ZUogMCYios Ui2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0As7gB6KKU9NL64qrnl7+jEEiK3R3QPZ3EzF2mVrRe8=; b=thu64Nr3ifDU9irw9fa+RwGGKNGFKGt4pGe13uplO/LCLMWEe8VDFFGmmBHUohgGct Hl+q6zhmHmS9FGIRiAF0eydcUP0UhuKInTQT2WKUCQj0/t1Af6yOwIgxfkaH2CrBMzxD /+9ggt/f6Wv6XW7naJZsIqV5Wn43g1fbw1xoTY3vHzaJaP2gf3VZnDrP5KcGL+VE++2T AJjI2Ly3Gt1OalfzsRFRPTVlT9CKkkQlj2IfeQva9XUrIbGroCzJrO/DlyaB38Z+AhfL GPddKr6qDG6x324UM9YXl+CLmkDyND5G1/hzXLpfnkVstvwnB5zO0a9DYjH4Wv+SQ3in 14xQ== X-Gm-Message-State: APjAAAWDA2zWwMOY+imN8EPiuWtdyMWWctJaAq5BDCqLh96a7kbph+K1 +kZiJsTuYtb5f9qjPDQGZIqVVM9juB0= X-Received: by 2002:a17:902:aa43:: with SMTP id c3mr5290659plr.11.1568829778950; Wed, 18 Sep 2019 11:02:58 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:50 -0700 Message-Id: <20190918180251.32003-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::641 Subject: [Qemu-devel] [RFC v2 4/5] exec: Adjust notdirty tracing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The memory_region_tb_read tracepoint is unreachable, since notdirty is supposed to apply only to writes. The memory_region_tb_write tracepoint is mis-named, because notdirty is not only used for TB invalidation. It is also used for e.g. VGA RAM updates and migration. Replace memory_region_tb_write with memory_notdirty_write_access, and place it in memory_notdirty_write_prepare where it can catch all of the instances. Add memory_notdirty_set_dirty to log when we no longer intercept writes to a page. Signed-off-by: Richard Henderson --- v2: Rename new tracepoints as suggested (David) --- exec.c | 3 +++ memory.c | 4 ---- trace-events | 4 ++-- 3 files changed, 5 insertions(+), 6 deletions(-) -- 2.17.1 Reviewed-by: David Hildenbrand diff --git a/exec.c b/exec.c index 8b998974f8..5f2587b621 100644 --- a/exec.c +++ b/exec.c @@ -2755,6 +2755,8 @@ void memory_notdirty_write_prepare(NotDirtyInfo *ndi, ndi->size = size; ndi->pages = NULL; + trace_memory_notdirty_write_access(mem_vaddr, ram_addr, size); + assert(tcg_enabled()); if (!cpu_physical_memory_get_dirty_flag(ram_addr, DIRTY_MEMORY_CODE)) { ndi->pages = page_collection_lock(ram_addr, ram_addr + size); @@ -2779,6 +2781,7 @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi) /* we remove the notdirty callback only if the code has been flushed */ if (!cpu_physical_memory_is_clean(ndi->ram_addr)) { + trace_memory_notdirty_set_dirty(ndi->mem_vaddr); tlb_set_dirty(ndi->cpu, ndi->mem_vaddr); } } diff --git a/memory.c b/memory.c index b9dd6b94ca..57c44c97db 100644 --- a/memory.c +++ b/memory.c @@ -438,7 +438,6 @@ static MemTxResult memory_region_read_accessor(MemoryRegion *mr, /* Accesses to code which has previously been translated into a TB show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size); @@ -465,7 +464,6 @@ static MemTxResult memory_region_read_with_attrs_accessor(MemoryRegion *mr, /* Accesses to code which has previously been translated into a TB show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_read(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size); @@ -490,7 +488,6 @@ static MemTxResult memory_region_write_accessor(MemoryRegion *mr, /* Accesses to code which has previously been translated into a TB show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size); @@ -515,7 +512,6 @@ static MemTxResult memory_region_write_with_attrs_accessor(MemoryRegion *mr, /* Accesses to code which has previously been translated into a TB show * up in the MMIO path, as accesses to the io_mem_notdirty * MemoryRegion. */ - trace_memory_region_tb_write(get_cpu_index(), addr, tmp, size); } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size); diff --git a/trace-events b/trace-events index 823a4ae64e..20821ba545 100644 --- a/trace-events +++ b/trace-events @@ -52,14 +52,14 @@ dma_map_wait(void *dbs) "dbs=%p" find_ram_offset(uint64_t size, uint64_t offset) "size: 0x%" PRIx64 " @ 0x%" PRIx64 find_ram_offset_loop(uint64_t size, uint64_t candidate, uint64_t offset, uint64_t next, uint64_t mingap) "trying size: 0x%" PRIx64 " @ 0x%" PRIx64 ", offset: 0x%" PRIx64" next: 0x%" PRIx64 " mingap: 0x%" PRIx64 ram_block_discard_range(const char *rbname, void *hva, size_t length, bool need_madvise, bool need_fallocate, int ret) "%s@%p + 0x%zx: madvise: %d fallocate: %d ret: %d" +memory_notdirty_write_access(uint64_t vaddr, uint64_t ram_addr, unsigned size) "0x%" PRIx64 " ram_addr 0x%" PRIx64 " size %u" +memory_notdirty_set_dirty(uint64_t vaddr) "0x%" PRIx64 # memory.c memory_region_ops_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_ops_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_subpage_read(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_subpage_write(int cpu_index, void *mr, uint64_t offset, uint64_t value, unsigned size) "cpu %d mr %p offset 0x%"PRIx64" value 0x%"PRIx64" size %u" -memory_region_tb_read(int cpu_index, uint64_t addr, uint64_t value, unsigned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u" -memory_region_tb_write(int cpu_index, uint64_t addr, uint64_t value, unsigned size) "cpu %d addr 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_ram_device_read(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u" memory_region_ram_device_write(int cpu_index, void *mr, uint64_t addr, uint64_t value, unsigned size) "cpu %d mr %p addr 0x%"PRIx64" value 0x%"PRIx64" size %u" flatview_new(void *view, void *root) "%p (root %p)" From patchwork Wed Sep 18 18:02:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 173996 Delivered-To: patch@linaro.org Received: by 2002:ac9:19ad:0:0:0:0:0 with SMTP id d45csp18328oce; Wed, 18 Sep 2019 11:10:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqxLIKcVwd74oM1AUazfedrXpmA3LtOqnoGdMJjR3imhOE0fr5ciIO2Vx78SsQutvFZ4I/K0 X-Received: by 2002:ae9:ee0d:: with SMTP id i13mr5275597qkg.417.1568830233286; Wed, 18 Sep 2019 11:10:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1568830233; cv=none; d=google.com; s=arc-20160816; b=VV/m2+tHuki6DjthMjRGK8pvdzR2WMZVqn0tg/4Xr91cyJS9yforVjRT6p0om7Nqtm psWswZ5TikU+4RUBfo3VF+IpWmyvSBgoJTf1sCTDOQz8bgB2V12OtkOR4G8WZDPzeJXj BMFA/g8R0XFXI+48Nu+FNTYaQVx+DsfIXyiZdPrTzEkXg/FfYhL9TWq6/NGlbXtskfV6 D3MC9QxfSATKCDqS0Lp2bLsi5nXAoULNDfdr4JT0zsMpLqfO1VUie5WbgjJCQQr2cN5w EZQqPMa5R21CqIPFqVGt8IXGBpbYnIxHzmS4Y1i8DJtl6fkK5UQNshGSvSB+mu5N1NDk Duow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=XLbwjWXO2o10cL5F/+EEPP36UeeSrQvwCcEMZEHp8Do=; b=MjSCF9PE49p8qdqcDoJ+PNNgBO38u+7FPANjVOXEcI69Mhd7TS+DiIK+l4zGXSthko InYb406LhraF+ET3B/z5z1FHh1c66QcxDsIR1Kv3mPJqvIRr4drfDNaXYKc6gxUb7c3v gEsKgc2p86kQxTeupi+vu+XaJQwZnyck2Ojw/k1HSGVeSh3rO0Az8MPkmg1IGeubN74f uWhEbf3LnKRVTkb/BTowfyrjtg0O7JQxfhozCT+O45lGangNEBtk4EdewZK9JcTpbzDD IUuRhCYDLvCgdNzhX2vMcXkMU09ykzkXvlzgF4n72LpxdnM2jB9VBE0IcN+3O8CrDj8j FDvA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="nTU/ED6P"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a40si4487241qtk.374.2019.09.18.11.10.33 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 18 Sep 2019 11:10:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="nTU/ED6P"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34030 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAePQ-0008Nu-Nl for patch@linaro.org; Wed, 18 Sep 2019 14:10:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:45914) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1iAeIG-000240-Va for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1iAeIE-0000Ui-Ct for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:08 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:43955) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1iAeIA-0000TB-Ut for qemu-devel@nongnu.org; Wed, 18 Sep 2019 14:03:04 -0400 Received: by mail-pl1-x644.google.com with SMTP id 4so308112pld.10 for ; Wed, 18 Sep 2019 11:03:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XLbwjWXO2o10cL5F/+EEPP36UeeSrQvwCcEMZEHp8Do=; b=nTU/ED6P7tx9oR8mjY6GuoWqAtdeLxB+hvbHKfkaBosbes8Q2JsZ9P27HwAzIOUB9P c5oC+NZDJn1QnwOkApyWRoX8lVkH5PW7Iggct9LXEah6C+V6UYHTt9/TUR9PY8HwAoTS RDzkr2aZDNsp3a1YjLSGDcMi5G8LsSi5qmdFG100qe0Q0o/tL8GBt3B5hXqeOcRRP7nx HAJdLVQUFnGKbIxOyTp7M3JVTVkbWjXrbqOJ5E3B6ctiKZQ0ZSu6Jezb6cS2lVCQ0fnT qM2s5QPtIlCDZtnVf71qmk/oTQTjV80HdfidOyRYjBkgB3mV/YcAJzlRJBVhWmOqadRg uwfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XLbwjWXO2o10cL5F/+EEPP36UeeSrQvwCcEMZEHp8Do=; b=WVKKa7fXaLLom6QGYcpEsHga2HswrjL4C6I+Efn8LhWU0kh+yVp0G5VsF8meQXRT0E Ri5ZERlodtC0UGHf6CE9cjrGPXgVnWNeMWRmhhQ6hICRidLlPzeE3PIKuN7mO8T0fRsQ qhUY6iHCy8YfvuW8CcZuCJcz1tzl/UL52FqNpYbETjXoqf1M7CI/dXb0NFX/+OmAP0/p uXCsF0vFryqjsBbGxCuGzmW3qViqpvbkTPMDOXH+/unXDuS2YSrBdiyQL+zSGBbHxyOM lKokD38OgS/NDS7Ivw9mK+Y0DsBMsvaLz/e4BEfZG5CWRG4t998s4CK02ZpJaileSFnW 6rww== X-Gm-Message-State: APjAAAU4SF9OnaJ88q+AnSPvZHccTKj/OJuL/GpAmauBVtZGQXpBif8n h4tfhlRChgqIy0ytB2sYmZFnnSww6v8= X-Received: by 2002:a17:902:8bca:: with SMTP id r10mr5407494plo.338.1568829780263; Wed, 18 Sep 2019 11:03:00 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id b16sm13319518pfb.54.2019.09.18.11.02.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 18 Sep 2019 11:02:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 18 Sep 2019 11:02:51 -0700 Message-Id: <20190918180251.32003-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190918180251.32003-1-richard.henderson@linaro.org> References: <20190918180251.32003-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::644 Subject: [Qemu-devel] [RFC v2 5/5] cputlb: Move NOTDIRTY handling from I/O path to TLB path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pbonzini@redhat.com, alex.bennee@linaro.org, stefanha@redhat.com, david@redhat.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Pages that we want to track for NOTDIRTY are RAM. We do not really need to go through the I/O path to handle them. Signed-off-by: Richard Henderson --- include/exec/cpu-common.h | 1 - accel/tcg/cputlb.c | 23 ++++++++++++++--- exec.c | 54 +++------------------------------------ memory.c | 16 ------------ 4 files changed, 23 insertions(+), 71 deletions(-) -- 2.17.1 diff --git a/include/exec/cpu-common.h b/include/exec/cpu-common.h index f7dbe75fbc..06c60c82be 100644 --- a/include/exec/cpu-common.h +++ b/include/exec/cpu-common.h @@ -101,7 +101,6 @@ void qemu_flush_coalesced_mmio_buffer(void); void cpu_flush_icache_range(hwaddr start, hwaddr len); extern struct MemoryRegion io_mem_rom; -extern struct MemoryRegion io_mem_notdirty; typedef int (RAMBlockIterFunc)(RAMBlock *rb, void *opaque); diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 7301f9e699..c6c66b40bb 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -904,7 +904,7 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry, mr = section->mr; mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; cpu->mem_io_pc = retaddr; - if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { + if (mr != &io_mem_rom && !cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } @@ -945,7 +945,7 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry, section = iotlb_to_section(cpu, iotlbentry->addr, iotlbentry->attrs); mr = section->mr; mr_offset = (iotlbentry->addr & TARGET_PAGE_MASK) + addr; - if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { + if (mr != &io_mem_rom && !cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } cpu->mem_io_vaddr = addr; @@ -1607,7 +1607,7 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, } /* Handle I/O access. */ - if (likely(tlb_addr & (TLB_MMIO | TLB_NOTDIRTY))) { + if (tlb_addr & TLB_MMIO) { io_writex(env, iotlbentry, mmu_idx, val, addr, retaddr, op ^ (tlb_addr & TLB_BSWAP ? MO_BSWAP : 0)); return; @@ -1615,6 +1615,23 @@ store_helper(CPUArchState *env, target_ulong addr, uint64_t val, haddr = (void *)((uintptr_t)addr + entry->addend); + /* Handle clean RAM pages. */ + if (tlb_addr & TLB_NOTDIRTY) { + NotDirtyInfo ndi; + + memory_notdirty_write_prepare(&ndi, env_cpu(env), addr, + addr + iotlbentry->addr, size); + + if (unlikely(tlb_addr & TLB_BSWAP)) { + direct_swap(haddr, val); + } else { + direct(haddr, val); + } + + memory_notdirty_write_complete(&ndi); + return; + } + if (unlikely(tlb_addr & TLB_BSWAP)) { direct_swap(haddr, val); } else { diff --git a/exec.c b/exec.c index 5f2587b621..4600292ee2 100644 --- a/exec.c +++ b/exec.c @@ -88,7 +88,7 @@ static MemoryRegion *system_io; AddressSpace address_space_io; AddressSpace address_space_memory; -MemoryRegion io_mem_rom, io_mem_notdirty; +MemoryRegion io_mem_rom; static MemoryRegion io_mem_unassigned; #endif @@ -191,8 +191,7 @@ typedef struct subpage_t { } subpage_t; #define PHYS_SECTION_UNASSIGNED 0 -#define PHYS_SECTION_NOTDIRTY 1 -#define PHYS_SECTION_ROM 2 +#define PHYS_SECTION_ROM 1 static void io_mem_init(void); static void memory_map_init(void); @@ -1473,9 +1472,7 @@ hwaddr memory_region_section_get_iotlb(CPUState *cpu, if (memory_region_is_ram(section->mr)) { /* Normal RAM. */ iotlb = memory_region_get_ram_addr(section->mr) + xlat; - if (!section->readonly) { - iotlb |= PHYS_SECTION_NOTDIRTY; - } else { + if (section->readonly) { iotlb |= PHYS_SECTION_ROM; } } else { @@ -2786,42 +2783,6 @@ void memory_notdirty_write_complete(NotDirtyInfo *ndi) } } -/* Called within RCU critical section. */ -static void notdirty_mem_write(void *opaque, hwaddr ram_addr, - uint64_t val, unsigned size) -{ - NotDirtyInfo ndi; - - memory_notdirty_write_prepare(&ndi, current_cpu, current_cpu->mem_io_vaddr, - ram_addr, size); - - stn_p(qemu_map_ram_ptr(NULL, ram_addr), size, val); - memory_notdirty_write_complete(&ndi); -} - -static bool notdirty_mem_accepts(void *opaque, hwaddr addr, - unsigned size, bool is_write, - MemTxAttrs attrs) -{ - return is_write; -} - -static const MemoryRegionOps notdirty_mem_ops = { - .write = notdirty_mem_write, - .valid.accepts = notdirty_mem_accepts, - .endianness = DEVICE_NATIVE_ENDIAN, - .valid = { - .min_access_size = 1, - .max_access_size = 8, - .unaligned = false, - }, - .impl = { - .min_access_size = 1, - .max_access_size = 8, - .unaligned = false, - }, -}; - /* Generate a debug exception if a watchpoint has been hit. */ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len, MemTxAttrs attrs, int flags, uintptr_t ra) @@ -3051,13 +3012,6 @@ static void io_mem_init(void) NULL, NULL, UINT64_MAX); memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL, NULL, UINT64_MAX); - - /* io_mem_notdirty calls tb_invalidate_phys_page_fast, - * which can be called without the iothread mutex. - */ - memory_region_init_io(&io_mem_notdirty, NULL, ¬dirty_mem_ops, NULL, - NULL, UINT64_MAX); - memory_region_clear_global_locking(&io_mem_notdirty); } AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv) @@ -3067,8 +3021,6 @@ AddressSpaceDispatch *address_space_dispatch_new(FlatView *fv) n = dummy_section(&d->map, fv, &io_mem_unassigned); assert(n == PHYS_SECTION_UNASSIGNED); - n = dummy_section(&d->map, fv, &io_mem_notdirty); - assert(n == PHYS_SECTION_NOTDIRTY); n = dummy_section(&d->map, fv, &io_mem_rom); assert(n == PHYS_SECTION_ROM); diff --git a/memory.c b/memory.c index 57c44c97db..a99b8c0767 100644 --- a/memory.c +++ b/memory.c @@ -434,10 +434,6 @@ static MemTxResult memory_region_read_accessor(MemoryRegion *mr, tmp = mr->ops->read(mr->opaque, addr, size); if (mr->subpage) { trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, size); - } else if (mr == &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size); @@ -460,10 +456,6 @@ static MemTxResult memory_region_read_with_attrs_accessor(MemoryRegion *mr, r = mr->ops->read_with_attrs(mr->opaque, addr, &tmp, size, attrs); if (mr->subpage) { trace_memory_region_subpage_read(get_cpu_index(), mr, addr, tmp, size); - } else if (mr == &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_READ_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_read(get_cpu_index(), mr, abs_addr, tmp, size); @@ -484,10 +476,6 @@ static MemTxResult memory_region_write_accessor(MemoryRegion *mr, if (mr->subpage) { trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, size); - } else if (mr == &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size); @@ -508,10 +496,6 @@ static MemTxResult memory_region_write_with_attrs_accessor(MemoryRegion *mr, if (mr->subpage) { trace_memory_region_subpage_write(get_cpu_index(), mr, addr, tmp, size); - } else if (mr == &io_mem_notdirty) { - /* Accesses to code which has previously been translated into a TB show - * up in the MMIO path, as accesses to the io_mem_notdirty - * MemoryRegion. */ } else if (TRACE_MEMORY_REGION_OPS_WRITE_ENABLED) { hwaddr abs_addr = memory_region_to_absolute_addr(mr, addr); trace_memory_region_ops_write(get_cpu_index(), mr, abs_addr, tmp, size);