From patchwork Mon Jan 7 07:15:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 154872 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp3192671ljp; Sun, 6 Jan 2019 23:15:29 -0800 (PST) X-Google-Smtp-Source: AFSGD/Wu20XDktnibK+guoe7IbCx4/Z8BMyzmCs1VSmfN8HUputAwINH9M+dOQRqch7UBvqGjt64 X-Received: by 2002:a62:d148:: with SMTP id t8mr63465590pfl.52.1546845329599; Sun, 06 Jan 2019 23:15:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1546845329; cv=none; d=google.com; s=arc-20160816; b=vNe+mkRv9UiSgUIHxpJVsknqCFmwZHHZEwj78jBhCmFp14au33QpEBlrCw6PxtHBYF gzwrSkDvTAvzhwFgE8kAQngKQzuO1xS1h8n71YIo/+7jafU3WCugwG3u3HNKpdlU6n5f twyAgNW8wacKr7hJAvCSau8dEAPVWKY2EGKbp5vkF1v+HeXrB7R/mMpRcA7imnA87FfK G3I3Cam92P91cgcrafHdrGrEJdyyyXBkQhMqK7stj+jS0HoZgzi8IPLHLGvDPzf6DOOB HbirKTZdrCJTC8t2kYEUBNh7Rr/H1m5GlOiNRIWHIIUThcNDvJI+mvDTcttBBv1ViS2y 7KHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:list-subscribe:list-help :list-post:list-archive:list-unsubscribe:list-id:precedence:subject :mime-version:references:in-reply-to:message-id:date:to:from :dkim-signature:delivered-to; bh=Z6OsdVz7aknIikqWm6n/A2wF3dLJsAIQqOuYFii3e+0=; b=vAp2KIdUDhiP2scAdKsQuAeS62iBgpbC7C4XK+blIJ4XU7tjdPuFQAwpEzgS82Q1gA 3TvyzENONB+MSC66Fw+Z9Tx4i/vibbi86ekd2BfvseD1CCZKXwfDs0RT8GsBExzSrpJe 0t0YcnzSw+3ABVQ7f6ZTOOrEbgtX53B3+TqXCzn9hEF5LEmLzJ8hKtk5zAB8DNuDEmB7 vHBcfYjcCiubhFXNyv5mye7ykkjfEXHUaIAGes08as7u94Khj8utNKGLvfKR5a6CA65+ WUTZGg75fdFCBrWIXMDRQbBTEtjS+lP/E0JiJsYvWb9GqLO6IHPYxAq5WbjI82PlwStA +jgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=kOE8iKDl; spf=pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 2001:19d0:306:5::1 as permitted sender) smtp.mailfrom=edk2-devel-bounces@lists.01.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from ml01.01.org (ml01.01.org. [2001:19d0:306:5::1]) by mx.google.com with ESMTPS id k64si59248164pge.7.2019.01.06.23.15.29 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Jan 2019 23:15:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 2001:19d0:306:5::1 as permitted sender) client-ip=2001:19d0:306:5::1; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=kOE8iKDl; spf=pass (google.com: best guess record for domain of edk2-devel-bounces@lists.01.org designates 2001:19d0:306:5::1 as permitted sender) smtp.mailfrom=edk2-devel-bounces@lists.01.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id E42B3211B1134; Sun, 6 Jan 2019 23:15:25 -0800 (PST) X-Original-To: edk2-devel@lists.01.org Delivered-To: edk2-devel@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=2a00:1450:4864:20::543; helo=mail-ed1-x543.google.com; envelope-from=ard.biesheuvel@linaro.org; receiver=edk2-devel@lists.01.org Received: from mail-ed1-x543.google.com (mail-ed1-x543.google.com [IPv6:2a00:1450:4864:20::543]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8E48B21B02822 for ; Sun, 6 Jan 2019 23:15:24 -0800 (PST) Received: by mail-ed1-x543.google.com with SMTP id d39so36889235edb.12 for ; Sun, 06 Jan 2019 23:15:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eE/Fh6bjGXWm4tQ7Uaw5T3VBqcOgnBFtP+IC2qNG/tc=; b=kOE8iKDlowyUfcTusOmDdFprOfeoyoXpYD6NIrwhjySIZmAIr2kSsnj6sDRjKtBwar jpACDkmocynoILqGcl7biaW2v00L7ADYEmYQYYwWS5H4gtWjluRlnuHmDxnzsd+Bmscu 7KbOp7ZCRVs6dZxzBf9rDsl3LKbSc/kc1POOE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eE/Fh6bjGXWm4tQ7Uaw5T3VBqcOgnBFtP+IC2qNG/tc=; b=JWiaJbljXyuk23TqASGa7dxyEph6J+/KotvmIW7KFuosLURrpYJSfda/0qLbjm7lxp oAF4GblrfkfT2dqLw6Qy1QSAW7zX8WSCttJVF7XS5TCReIcOOpymgExB1rJ46ZezZtnD AaUhPp1QCbM2+c8QaszK0c8uOlLABin7J7/GrjGJ/YkNGCSjhLi48h33lZXt78kJYoU1 uh9/ynQ2r89L4mJ4LQoO2YtSNXJbGqmIOfAP6GrdVJsiQ1eoqYUQvV9huJ4C/G+4TkuW G2GddtVAh/jXt0rHJQKxkdpvvwByYt1JFpaEmbP3WgpkCpZSs9QZHcPq9MwJsfWph9d5 MQ2Q== X-Gm-Message-State: AA+aEWZ8TXvve9LgpAFN3vrZJIc4RC3h2sBfEgRgsAJPLZlxTTsqciEP BTivzP6u4m/H+9/hpSyZ0q8Wabxk6mwcNQ== X-Received: by 2002:a50:ae64:: with SMTP id c91mr54392912edd.222.1546845322795; Sun, 06 Jan 2019 23:15:22 -0800 (PST) Received: from chuckie.home ([2a01:cb1d:112:6f00:58f2:776e:9e23:a7ca]) by smtp.gmail.com with ESMTPSA id t9sm30263693edd.25.2019.01.06.23.15.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 06 Jan 2019 23:15:22 -0800 (PST) From: Ard Biesheuvel To: edk2-devel@lists.01.org Date: Mon, 7 Jan 2019 08:15:01 +0100 Message-Id: <20190107071504.2431-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190107071504.2431-1-ard.biesheuvel@linaro.org> References: <20190107071504.2431-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Subject: [edk2] [PATCH 2/5] ArmPkg/ArmMmuLib AARCH64: get rid of needless TLB invalidation X-BeenThere: edk2-devel@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: EDK II Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: edk2-devel-bounces@lists.01.org Sender: "edk2-devel" Currently, we always invalidate the TLBs entirely after making any modification to the page tables. Now that we have introduced strict memory permissions in quite a number of places, such modifications occur much more often, and it is better for performance to flush only those TLB entries that are actually affected by the changes. Contributed-under: TianoCore Contribution Agreement 1.1 Signed-off-by: Ard Biesheuvel --- ArmPkg/Include/Library/ArmMmuLib.h | 3 ++- ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S | 6 +++--- ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c | 16 +++++++--------- ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S | 14 ++++++++------ 4 files changed, 20 insertions(+), 19 deletions(-) -- 2.20.1 _______________________________________________ edk2-devel mailing list edk2-devel@lists.01.org https://lists.01.org/mailman/listinfo/edk2-devel Reviewed-by: Leif Lindholm Reviewed-by: Leif Lindholm diff --git a/ArmPkg/Include/Library/ArmMmuLib.h b/ArmPkg/Include/Library/ArmMmuLib.h index fb7fd006417c..d2725810f1c6 100644 --- a/ArmPkg/Include/Library/ArmMmuLib.h +++ b/ArmPkg/Include/Library/ArmMmuLib.h @@ -59,7 +59,8 @@ VOID EFIAPI ArmReplaceLiveTranslationEntry ( IN UINT64 *Entry, - IN UINT64 Value + IN UINT64 Value, + IN UINT64 Address ); EFI_STATUS diff --git a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S index b7173e00b039..175fb58206b6 100644 --- a/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S +++ b/ArmPkg/Library/ArmLib/AArch64/ArmLibSupport.S @@ -124,15 +124,15 @@ ASM_FUNC(ArmSetMAIR) // IN VOID *MVA // X1 // ); ASM_FUNC(ArmUpdateTranslationTableEntry) - dc civac, x0 // Clean and invalidate data line - dsb sy + dsb nshst + lsr x1, x1, #12 EL1_OR_EL2_OR_EL3(x0) 1: tlbi vaae1, x1 // TLB Invalidate VA , EL1 b 4f 2: tlbi vae2, x1 // TLB Invalidate VA , EL2 b 4f 3: tlbi vae3, x1 // TLB Invalidate VA , EL3 -4: dsb sy +4: dsb nsh isb ret diff --git a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c index d66df3e17a02..e1fabfcbea14 100644 --- a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c +++ b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibCore.c @@ -129,13 +129,14 @@ STATIC VOID ReplaceLiveEntry ( IN UINT64 *Entry, - IN UINT64 Value + IN UINT64 Value, + IN UINT64 Address ) { if (!ArmMmuEnabled ()) { *Entry = Value; } else { - ArmReplaceLiveTranslationEntry (Entry, Value); + ArmReplaceLiveTranslationEntry (Entry, Value, Address); } } @@ -296,7 +297,8 @@ GetBlockEntryListFromAddress ( // Fill the BlockEntry with the new TranslationTable ReplaceLiveEntry (BlockEntry, - ((UINTN)TranslationTable & TT_ADDRESS_MASK_DESCRIPTION_TABLE) | TableAttributes | TT_TYPE_TABLE_ENTRY); + (UINTN)TranslationTable | TableAttributes | TT_TYPE_TABLE_ENTRY, + RegionStart); } } else { if (IndexLevel != PageLevel) { @@ -375,6 +377,8 @@ UpdateRegionMapping ( *BlockEntry &= BlockEntryMask; *BlockEntry |= (RegionStart & TT_ADDRESS_MASK_BLOCK_ENTRY) | Attributes | Type; + ArmUpdateTranslationTableEntry (BlockEntry, (VOID *)RegionStart); + // Go to the next BlockEntry RegionStart += BlockEntrySize; RegionLength -= BlockEntrySize; @@ -487,9 +491,6 @@ ArmSetMemoryAttributes ( return Status; } - // Invalidate all TLB entries so changes are synced - ArmInvalidateTlb (); - return EFI_SUCCESS; } @@ -512,9 +513,6 @@ SetMemoryRegionAttribute ( return Status; } - // Invalidate all TLB entries so changes are synced - ArmInvalidateTlb (); - return EFI_SUCCESS; } diff --git a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S index 90192df24f55..d40c19b2e3e5 100644 --- a/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S +++ b/ArmPkg/Library/ArmMmuLib/AArch64/ArmMmuLibReplaceEntry.S @@ -32,11 +32,12 @@ dmb sy dc ivac, x0 - // flush the TLBs + // flush translations for the target address from the TLBs + lsr x2, x2, #12 .if \el == 1 - tlbi vmalle1 + tlbi vaae1, x2 .else - tlbi alle\el + tlbi vae\el, x2 .endif dsb sy @@ -48,12 +49,13 @@ //VOID //ArmReplaceLiveTranslationEntry ( // IN UINT64 *Entry, -// IN UINT64 Value +// IN UINT64 Value, +// IN UINT64 Address // ) ASM_FUNC(ArmReplaceLiveTranslationEntry) // disable interrupts - mrs x2, daif + mrs x4, daif msr daifset, #0xf isb @@ -69,7 +71,7 @@ ASM_FUNC(ArmReplaceLiveTranslationEntry) b 4f 3:__replace_entry 3 -4:msr daif, x2 +4:msr daif, x4 ret ASM_GLOBAL ASM_PFX(ArmReplaceLiveTranslationEntrySize)