From patchwork Tue Oct 23 07:02:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 149428 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp349374ljp; Tue, 23 Oct 2018 00:13:07 -0700 (PDT) X-Google-Smtp-Source: ACcGV63aNLuAW8EfJMS8TLZrfL15OWlqtC93+ArUxHkeOusONi+AUudaeoj3u2/lH3iEd3LuWyu6 X-Received: by 2002:a37:650b:: with SMTP id z11-v6mr44016910qkb.221.1540278787565; Tue, 23 Oct 2018 00:13:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540278787; cv=none; d=google.com; s=arc-20160816; b=mh5FqcAVzIv2Tf2W9U4aUAIDh0cjBVoWO4Q9KyaaT0XBhep7dcCxWcLyfrx3dGahz9 ALuoEY/wY7Sv7XK1fTmnB2cHUBQ/pOdr78U/PpZl4SM57VMJ3pFqN2WvHX9vqE3MubnE ZJh4YnX4FEg6oyX+WokLfGc/ZNldzVEalv7KCJTaR1tu9p7MIjzay5rZRbnJWnuaFvE/ 7NwnUxRhthh96L/GKTLJPRXw/mB3ug77DIPq+sSj1mJIB5q2TbN8dGOK3uhy/I3KDghd fIQOzKwCfmMUdWtQDgD6QJf0qYtwwZvMPGxVa4axT51LcAso8s4OmDFQIvW2LpRkp1+e /CSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=1hvg9z51OQOp7GSTXVGygTFja2i610PhoH7b+MvaDsk=; b=cNbboapI7M7ia8xyfaeJpt4RIaITbSwQeNHBZJ47dxYUUGZCBM9S2HJ0Q+sphc2eZQ BFnoh2ArRXaDpnTSSHJ+cjQn5McJPwrDDLLPJB/ZVJIWSaK5xsya3+L+DbCZzE7DHrZr Y+othh8I0op6mQDNeLMiNUY0bKmcrPl2Zxyk/TNcLPOrV3cpC0kOCVPJva7TiAe9+eBX vyE7BAFLUX7NPffx8q0cnjEN8f6RfTTZylLXr3trnx0a5qP7YAlbZu2WQ8Ee9KJgJbNb vbT4aLJp3iaOTSoLz0KZstcU4im4u/AyQOQjYiIjsKq65SzX/s16YRfaT62vHP9hu1NT 889Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=f7YoFWEE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id d6-v6si376025qvb.143.2018.10.23.00.13.07 for (version=TLS1 cipher=AES128-SHA bits=128/128); Tue, 23 Oct 2018 00:13:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=f7YoFWEE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38499 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEqsF-0005va-1W for patch@linaro.org; Tue, 23 Oct 2018 03:13:07 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59636) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gEqm4-0008Od-K8 for qemu-devel@nongnu.org; Tue, 23 Oct 2018 03:06:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gEqif-0003wx-ED for qemu-devel@nongnu.org; Tue, 23 Oct 2018 03:03:18 -0400 Received: from mail-wm1-x344.google.com ([2a00:1450:4864:20::344]:37048) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gEqif-0003tb-0g for qemu-devel@nongnu.org; Tue, 23 Oct 2018 03:03:13 -0400 Received: by mail-wm1-x344.google.com with SMTP id x19-v6so544984wmc.2 for ; Tue, 23 Oct 2018 00:03:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1hvg9z51OQOp7GSTXVGygTFja2i610PhoH7b+MvaDsk=; b=f7YoFWEET7er4ukkhJkxsohgLimk1Y9IOe2+sXLjVfdeYLycork9fw4il03HbfAd1A hLV+L/YGYwnG+QU0UugQAxxZ0hKkfyFyp4zR7ZnT1W2BwklLg9y/jzNlcdhOf8HDaxCr uQqUseyVqLvnUwuRckJP7Zx1V2Wk/ytz4kNcE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1hvg9z51OQOp7GSTXVGygTFja2i610PhoH7b+MvaDsk=; b=TPRl8jX1GBjuYld2yVgZR2xuOpEphgkcyIwLRmiXJ1qD2P+r6cQW7ltNanIkrjmjTx AEDu/gsGGkimdrbwKfv6+97wZubIY70W4HhZinLK12u5DRrs0fIW5RW8hRa+NHSjOkE4 9OFADhxMKenEGwgfZT9r7VTgqoaFWohztwrRiHmVnHO17pcOSDxNPi2cvcPEZ2pzoAKO j0HgTc0Ji2nSRuumEqrrwB77z7QmaU36RzAih6uKjCXTUzPNyWKu2OVoQ/JWNz6Jm6wW iA2PYzRdQ1WP+Yqyil9tqHXQoBvd6Pjbkl+96eksi8lDxwfkbzMvs/50Z+hEW1598Sgh //xA== X-Gm-Message-State: ABuFfog0f1ADgR/X5s3B99UleugzGsF5/1KAUsyFfknH2wCBhDXO50dO e+shKTpKkf9cSwI9cjmdf7eQ8wUx5iE= X-Received: by 2002:a1c:1a45:: with SMTP id a66-v6mr2574778wma.20.1540278191300; Tue, 23 Oct 2018 00:03:11 -0700 (PDT) Received: from cloudburst.twiddle.net.ASUS (host86-153-40-62.range86-153.btcentralplus.com. [86.153.40.62]) by smtp.gmail.com with ESMTPSA id t13-v6sm258355wrn.22.2018.10.23.00.03.10 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 Oct 2018 00:03:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Tue, 23 Oct 2018 08:02:52 +0100 Message-Id: <20181023070253.6407-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181023070253.6407-1-richard.henderson@linaro.org> References: <20181023070253.6407-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::344 Subject: [Qemu-devel] [PATCH 09/10] cputlb: Filter flushes on already clean tlbs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Especially for guests with large numbers of tlbs, like ARM or PPC, we may well not use all of them in between flush operations. Remember which tlbs have been used since the last flush, and avoid any useless flushing. Signed-off-by: Richard Henderson --- include/exec/cpu-defs.h | 7 ++++++- accel/tcg/cputlb.c | 35 +++++++++++++++++++++++++---------- 2 files changed, 31 insertions(+), 11 deletions(-) -- 2.17.2 diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h index c7b501d627..ca0fea8b27 100644 --- a/include/exec/cpu-defs.h +++ b/include/exec/cpu-defs.h @@ -166,7 +166,12 @@ typedef struct CPUTLBCommon { * mmu_idx may be discarded. Protected by tlb_c.lock. */ uint16_t pending_flush; - + /* + * Within dirty, for each bit N, modifications have been made to + * mmu_idx N since the last time that mmu_idx was flushed. + * Protected by tlb_c.lock. + */ + uint16_t dirty; /* * Statistics. These are not lock protected, but are read and * written atomically. This allows the monitor to print a snapshot diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 5480115cb4..c1b92083d4 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -79,6 +79,9 @@ void tlb_init(CPUState *cpu) CPUArchState *env = cpu->env_ptr; qemu_spin_init(&env->tlb_c.lock); + + /* Ensure that cpu_reset performs a full flush. */ + env->tlb_c.dirty = ALL_MMUIDX_BITS; } /* flush_all_helper: run fn across all cpus @@ -129,31 +132,40 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx) static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; - unsigned long mmu_idx_bitmask = data.host_int; - int mmu_idx; + uint16_t asked = data.host_int; + uint16_t all_dirty, work, to_clean; assert_cpu_is_self(cpu); - tlb_debug("mmu_idx:0x%04lx\n", mmu_idx_bitmask); + tlb_debug("mmu_idx:0x%04" PRIx16 "\n", asked); qemu_spin_lock(&env->tlb_c.lock); - env->tlb_c.pending_flush &= ~mmu_idx_bitmask; - for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - if (test_bit(mmu_idx, &mmu_idx_bitmask)) { - tlb_flush_one_mmuidx_locked(env, mmu_idx); - } + all_dirty = env->tlb_c.dirty; + to_clean = asked & all_dirty; + all_dirty &= ~to_clean; + env->tlb_c.dirty = all_dirty; + + for (work = to_clean; work != 0; work &= work - 1) { + int mmu_idx = ctz32(work); + tlb_flush_one_mmuidx_locked(env, mmu_idx); } + qemu_spin_unlock(&env->tlb_c.lock); cpu_tb_jmp_cache_clear(cpu); - if (mmu_idx_bitmask == ALL_MMUIDX_BITS) { + if (to_clean == ALL_MMUIDX_BITS) { atomic_set(&env->tlb_c.full_flush_count, env->tlb_c.full_flush_count + 1); } else { atomic_set(&env->tlb_c.part_flush_count, - env->tlb_c.part_flush_count + ctpop16(mmu_idx_bitmask)); + env->tlb_c.part_flush_count + ctpop16(to_clean)); + if (to_clean != asked) { + atomic_set(&env->tlb_c.elide_flush_count, + env->tlb_c.elide_flush_count + + ctpop16(asked & ~to_clean)); + } } } @@ -582,6 +594,9 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, */ qemu_spin_lock(&env->tlb_c.lock); + /* Note that the tlb is no longer clean. */ + env->tlb_c.dirty |= 1 << mmu_idx; + /* Make sure there's no cached translation for the new page. */ tlb_flush_vtlb_page_locked(env, mmu_idx, vaddr_page);