From patchwork Mon Dec 28 11:20:47 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 59012 Delivered-To: patch@linaro.org Received: by 10.112.130.2 with SMTP id oa2csp1648099lbb; Mon, 28 Dec 2015 03:25:17 -0800 (PST) X-Received: by 10.98.16.140 with SMTP id 12mr21984602pfq.160.1451301917559; Mon, 28 Dec 2015 03:25:17 -0800 (PST) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id gw4si19553942pac.129.2015.12.28.03.25.17 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 28 Dec 2015 03:25:17 -0800 (PST) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org; dkim=neutral (body hash did not verify) header.i=@linaro.org Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1aDVuK-0004Yt-SB; Mon, 28 Dec 2015 11:24:08 +0000 Received: from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1aDVrj-0002il-U0 for linux-arm-kernel@lists.infradead.org; Mon, 28 Dec 2015 11:21:31 +0000 Received: by mail-wm0-x234.google.com with SMTP id f206so4154306wmf.0 for ; Mon, 28 Dec 2015 03:21:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Aey/CVQNza4rgQ8PjPgER0dZxuoo63DxGaauNZ0iFUQ=; b=H3nJMUCO2SMHi3tUhiirqGgaw1xPrrScu1r8D7Fo+EWXdS3rqoPD+YewVyz9Ko87ZJ ECm+Nn8oVNNXHtOzUhL5ZrsQ7wQmYxzWjJebeHKpsJ/shwQ7nO+d4zZaQUfTxHVvNJSB 3BgPU1FyJHFILARkgYqCVP9Zv2Ei7IXjBCLKg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Aey/CVQNza4rgQ8PjPgER0dZxuoo63DxGaauNZ0iFUQ=; b=kmjy7bKPl3yEZHHOitiUULA6C79e05YNe8YlcUCVgrSDgOKLMwc8gjZLAhxH9S2mRT PyoaBsD2Jc5DxWNmptc9Aa0Zy0I6cceuq0uLyJ1kwB8Ga0rwCBDt8bPTDnrxS6IgY1YD uVfwvdereRfgw3kJCoj5LBNfYzQbS7WoilQObrwtJ4MRBuW2kMuyZyXuVFOoH34uzX6H HQurMfYYB5TXoiESB481dr7d875f46oZZl5QJtatUgomY+eeDjNOyEC/R+IAy3lMvTw7 vWzXSkDmmEluOW65LE7UWg4akTlWFNOt8wTah2uWsLUuifhaZ95KkUzCrmM8G3IvfXzt HK6w== X-Gm-Message-State: ALoCoQkP1g6PhUThBUQl3T9mfe8Gqj/rvID9fn/7WFE1x9k7xWU4/3R4PZZiZuoWIIRZic2tVzUpT350VwYKfoSaaFgNmz57fg== X-Received: by 10.194.204.166 with SMTP id kz6mr58315298wjc.174.1451301666379; Mon, 28 Dec 2015 03:21:06 -0800 (PST) Received: from localhost.localdomain (cag06-7-83-153-85-71.fbx.proxad.net. [83.153.85.71]) by smtp.gmail.com with ESMTPSA id i63sm22556235wmf.24.2015.12.28.03.21.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 28 Dec 2015 03:21:05 -0800 (PST) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, kernel-hardening@lists.openwall.com, will.deacon@arm.com, catalin.marinas@arm.com, mark.rutland@arm.com, leif.lindholm@linaro.org, keescook@chromium.org, lkml@vger.kernel.org Subject: [RFC PATCH 03/10] arm64: decouple early fixmap init from linear mapping Date: Mon, 28 Dec 2015 12:20:47 +0100 Message-Id: <1451301654-32019-4-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.5.0 In-Reply-To: <1451301654-32019-1-git-send-email-ard.biesheuvel@linaro.org> References: <1451301654-32019-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20151228_032128_408554_20A9E438 X-CRM114-Status: GOOD ( 14.99 ) X-Spam-Score: -2.7 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [2a00:1450:400c:c09:0:0:0:234 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: bhupesh.sharma@freescale.com, stuart.yoder@freescale.com, Ard Biesheuvel MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org Since the early fixmap page tables are populated using pages that are part of the static footprint of the kernel, they are covered by the initial kernel mapping, and we can refer to them without using __va/__pa translations, which are tied to the linear mapping. Instead, let's introduce __phys_to_kimg, which will be tied to the kernel virtual mapping, regardless of whether or not it intersects with the linear mapping. This will allow us to move the kernel out of the linear mapping in a subsequent patch. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/compiler.h | 2 + arch/arm64/kernel/vmlinux.lds.S | 12 +-- arch/arm64/mm/mmu.c | 85 ++++++++------------ 3 files changed, 42 insertions(+), 57 deletions(-) -- 2.5.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel diff --git a/arch/arm64/include/asm/compiler.h b/arch/arm64/include/asm/compiler.h index ee35fd0f2236..dd342af63673 100644 --- a/arch/arm64/include/asm/compiler.h +++ b/arch/arm64/include/asm/compiler.h @@ -27,4 +27,6 @@ */ #define __asmeq(x, y) ".ifnc " x "," y " ; .err ; .endif\n\t" +#define __pgdir __attribute__((section(".pgdir"),aligned(PAGE_SIZE))) + #endif /* __ASM_COMPILER_H */ diff --git a/arch/arm64/kernel/vmlinux.lds.S b/arch/arm64/kernel/vmlinux.lds.S index ced0dedcabcc..363c2f529951 100644 --- a/arch/arm64/kernel/vmlinux.lds.S +++ b/arch/arm64/kernel/vmlinux.lds.S @@ -160,11 +160,13 @@ SECTIONS BSS_SECTION(0, 0, 0) - . = ALIGN(PAGE_SIZE); - idmap_pg_dir = .; - . += IDMAP_DIR_SIZE; - swapper_pg_dir = .; - . += SWAPPER_DIR_SIZE; + .pgdir (NOLOAD) : ALIGN(PAGE_SIZE) { + idmap_pg_dir = .; + . += IDMAP_DIR_SIZE; + swapper_pg_dir = .; + . += SWAPPER_DIR_SIZE; + *(.pgdir) + } _end = .; diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 50b1de8e127b..3435c316d607 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -540,39 +540,11 @@ void vmemmap_free(unsigned long start, unsigned long end) } #endif /* CONFIG_SPARSEMEM_VMEMMAP */ -static pte_t bm_pte[PTRS_PER_PTE] __page_aligned_bss; -#if CONFIG_PGTABLE_LEVELS > 2 -static pmd_t bm_pmd[PTRS_PER_PMD] __page_aligned_bss; -#endif -#if CONFIG_PGTABLE_LEVELS > 3 -static pud_t bm_pud[PTRS_PER_PUD] __page_aligned_bss; -#endif - -static inline pud_t * fixmap_pud(unsigned long addr) -{ - pgd_t *pgd = pgd_offset_k(addr); - - BUG_ON(pgd_none(*pgd) || pgd_bad(*pgd)); - - return pud_offset(pgd, addr); -} - -static inline pmd_t * fixmap_pmd(unsigned long addr) -{ - pud_t *pud = fixmap_pud(addr); - - BUG_ON(pud_none(*pud) || pud_bad(*pud)); +static pte_t *__fixmap_pte; - return pmd_offset(pud, addr); -} - -static inline pte_t * fixmap_pte(unsigned long addr) +static inline pte_t *fixmap_pte(unsigned long addr) { - pmd_t *pmd = fixmap_pmd(addr); - - BUG_ON(pmd_none(*pmd) || pmd_bad(*pmd)); - - return pte_offset_kernel(pmd, addr); + return __fixmap_pte + pte_index(addr); } void __init early_fixmap_init(void) @@ -583,33 +555,42 @@ void __init early_fixmap_init(void) unsigned long addr = FIXADDR_START; pgd = pgd_offset_k(addr); - pgd_populate(&init_mm, pgd, bm_pud); - pud = pud_offset(pgd, addr); - pud_populate(&init_mm, pud, bm_pmd); - pmd = pmd_offset(pud, addr); - pmd_populate_kernel(&init_mm, pmd, bm_pte); +#if CONFIG_PGTABLE_LEVELS > 3 + if (pgd_none(*pgd)) { + static pud_t bm_pud[PTRS_PER_PUD] __pgdir; + + __pgd_populate(pgd, __pa(bm_pud), PUD_TYPE_TABLE); + memblock_reserve(__pa(bm_pud), sizeof(bm_pud)); + } + pud = (pud_t *)__phys_to_kimg(pud_offset_phys(pgd, addr)); +#else + pud = (pud_t *)pgd; +#endif +#if CONFIG_PGTABLE_LEVELS > 2 + if (pud_none(*pud)) { + static pmd_t bm_pmd[PTRS_PER_PMD] __pgdir; + + __pud_populate(pud, __pa(bm_pmd), PMD_TYPE_TABLE); + memblock_reserve(__pa(bm_pmd), sizeof(bm_pmd)); + } + pmd = (pmd_t *)__phys_to_kimg(pmd_offset_phys(pud, addr)); +#else + pmd = (pmd_t *)pud; +#endif + if (pmd_none(*pmd)) { + static pte_t bm_pte[PTRS_PER_PTE] __pgdir; + + __pmd_populate(pmd, __pa(bm_pte), PMD_TYPE_TABLE); + memblock_reserve(__pa(bm_pte), sizeof(bm_pte)); + } + __fixmap_pte = (pte_t *)__phys_to_kimg(pmd_page_paddr(*pmd)); /* * The boot-ioremap range spans multiple pmds, for which - * we are not preparted: + * we are not prepared: */ BUILD_BUG_ON((__fix_to_virt(FIX_BTMAP_BEGIN) >> PMD_SHIFT) != (__fix_to_virt(FIX_BTMAP_END) >> PMD_SHIFT)); - - if ((pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_BEGIN))) - || pmd != fixmap_pmd(fix_to_virt(FIX_BTMAP_END))) { - WARN_ON(1); - pr_warn("pmd %p != %p, %p\n", - pmd, fixmap_pmd(fix_to_virt(FIX_BTMAP_BEGIN)), - fixmap_pmd(fix_to_virt(FIX_BTMAP_END))); - pr_warn("fix_to_virt(FIX_BTMAP_BEGIN): %08lx\n", - fix_to_virt(FIX_BTMAP_BEGIN)); - pr_warn("fix_to_virt(FIX_BTMAP_END): %08lx\n", - fix_to_virt(FIX_BTMAP_END)); - - pr_warn("FIX_BTMAP_END: %d\n", FIX_BTMAP_END); - pr_warn("FIX_BTMAP_BEGIN: %d\n", FIX_BTMAP_BEGIN); - } } void __set_fixmap(enum fixed_addresses idx,