From patchwork Wed Oct 12 11:23:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 77547 Delivered-To: patch@linaro.org Received: by 10.140.97.247 with SMTP id m110csp401442qge; Wed, 12 Oct 2016 04:26:14 -0700 (PDT) X-Received: by 10.99.97.69 with SMTP id v66mr772089pgb.5.1476271574598; Wed, 12 Oct 2016 04:26:14 -0700 (PDT) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id l5si5408260pgj.98.2016.10.12.04.26.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 12 Oct 2016 04:26:14 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org; dmarc=fail (p=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1buHej-0004Ix-8z; Wed, 12 Oct 2016 11:25:05 +0000 Received: from mail-qk0-x22c.google.com ([2607:f8b0:400d:c09::22c]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1buHe4-00041V-My for linux-arm-kernel@lists.infradead.org; Wed, 12 Oct 2016 11:24:26 +0000 Received: by mail-qk0-x22c.google.com with SMTP id f128so24345831qkb.1 for ; Wed, 12 Oct 2016 04:24:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=t2oCDedJITL6TChNLX9gXZH26rdcYDtLkbGq0MbYns4=; b=KmFjmL3nQUqCh7kOvOtaeP6Zl9RCiRgAvHeT3vUMIlpvtWyuLOZ6/hzOCAJGfd7iCO d/1n1TKNJ6vDWAUxzCQBXgb2/5mht5gmdC+tJWpUkL/CXsbiavFf+ua6ufyzYSvuo1t5 qS3QL8NTnlxPKdBomAx4XdmJd++pmRKc1dgtU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=t2oCDedJITL6TChNLX9gXZH26rdcYDtLkbGq0MbYns4=; b=WxxhqIskSRiRE+CFyNVS/wX5wwt6LAtpotW7N6e70nXjw7kNoXB+L05iAZ3uzG8h5l tlODNFy17OQFngxdV7E8goPajk7i3S7uMKbepq6TAmHp7xhD7vAeqRNNsPnivgHj2Nds ESF20ZWSlGXgOs7I5sgmM5ZxPN6kBxRNBw3quonMj1hkLIz0lrBLNeVShwa+NmeykNAj 8yoRZi3obkg8Rl7tHWJZ5ER2sP63rkoNsl4x2CU2yE2MNczWh/hiEIExjrCsguVl34V4 rO8cslI2vuyw3ZGYQ2lfbohPhs3GYEbfKfM5qM5ySrzVe5ocB7hukoEvrM7ZAaEUbPkP Ad2w== X-Gm-Message-State: AA6/9RnmEpym4TSHhZugG6HkVfrUjL6JunD9NsU7FC7+GGQgT0qPsdvyI3HZYLQsYlk++ZPp X-Received: by 10.194.62.205 with SMTP id a13mr716963wjs.89.1476271443373; Wed, 12 Oct 2016 04:24:03 -0700 (PDT) Received: from localhost.localdomain ([196.67.25.208]) by smtp.gmail.com with ESMTPSA id jt8sm12034690wjc.33.2016.10.12.04.24.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 12 Oct 2016 04:24:02 -0700 (PDT) From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, catalin.marinas@arm.com, will.deacon@arm.com Subject: [PATCH v3 4/5] arm64: mm: support additional contiguous kernel mapping region sizes Date: Wed, 12 Oct 2016 12:23:44 +0100 Message-Id: <1476271425-19401-5-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1476271425-19401-1-git-send-email-ard.biesheuvel@linaro.org> References: <1476271425-19401-1-git-send-email-ard.biesheuvel@linaro.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20161012_042424_925692_6023C915 X-CRM114-Status: GOOD ( 16.53 ) X-Spam-Score: -2.7 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [2607:f8b0:400d:c09:0:0:0:22c listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: steve.capper@linaro.org, jeremy.linton@arm.com, Ard Biesheuvel MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org Extend the basic support for kernel mappings using contiguous regions by adding support for contiguous PUDs (4k granule only), either as a discrete level or folded into the PGDs. In the same way, handle folded PMDs so that contiguous PMDs (for 16k and 64k granule kernels) will work as expected for 2 levels of translation as well. Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/pgtable-hwdef.h | 6 +++ arch/arm64/mm/mmu.c | 40 +++++++++++++++++++- 2 files changed, 44 insertions(+), 2 deletions(-) -- 2.7.4 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h index eb0c2bd90de9..4192072af932 100644 --- a/arch/arm64/include/asm/pgtable-hwdef.h +++ b/arch/arm64/include/asm/pgtable-hwdef.h @@ -93,12 +93,15 @@ #ifdef CONFIG_ARM64_64K_PAGES #define CONT_PTE_SHIFT 5 #define CONT_PMD_SHIFT 5 +#define CONT_PUD_SHIFT 0 #elif defined(CONFIG_ARM64_16K_PAGES) #define CONT_PTE_SHIFT 7 #define CONT_PMD_SHIFT 5 +#define CONT_PUD_SHIFT 0 #else #define CONT_PTE_SHIFT 4 #define CONT_PMD_SHIFT 4 +#define CONT_PUD_SHIFT 4 #endif #define CONT_PTES (1 << CONT_PTE_SHIFT) @@ -107,6 +110,9 @@ #define CONT_PMDS (1 << CONT_PMD_SHIFT) #define CONT_PMD_SIZE (CONT_PMDS * PMD_SIZE) #define CONT_PMD_MASK (~(CONT_PMD_SIZE - 1)) +#define CONT_PUDS (1 << CONT_PUD_SHIFT) +#define CONT_PUD_SIZE (CONT_PUDS * PUD_SIZE) +#define CONT_PUD_MASK (~(CONT_PUD_SIZE - 1)) /* the the numerical offset of the PTE within a range of CONT_PTES */ #define CONT_RANGE_OFFSET(addr) (((addr)>>PAGE_SHIFT)&(CONT_PTES-1)) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 40be4979102d..0e0eca45b54a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -233,6 +233,7 @@ static void alloc_init_pud(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t (*pgtable_alloc)(void), bool page_mappings_only) { + pgprot_t __prot = prot; pud_t *pud; unsigned long next; @@ -254,7 +255,19 @@ static void alloc_init_pud(pgd_t *pgd, unsigned long addr, unsigned long end, * For 4K granule only, attempt to put down a 1GB block */ if (use_1G_block(addr, next, phys) && !page_mappings_only) { - pud_set_huge(pud, phys, prot); + /* + * Set the contiguous bit for the subsequent group of + * PUDs if its size and alignment are appropriate. + */ + if (((addr | phys) & ~CONT_PUD_MASK) == 0) { + if (end - addr >= CONT_PUD_SIZE) + __prot = __pgprot(pgprot_val(prot) | + PTE_CONT); + else + __prot = prot; + } + + pud_set_huge(pud, phys, __prot); /* * After the PUD entry has been populated once, we @@ -284,6 +297,7 @@ static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, { unsigned long addr, length, end, next; pgd_t *pgd = pgd_offset_raw(pgdir, virt); + pgprot_t __prot = prot; /* * If the virtual and physical address don't have the same offset @@ -299,7 +313,29 @@ static void __create_pgd_mapping(pgd_t *pgdir, phys_addr_t phys, end = addr + length; do { next = pgd_addr_end(addr, end); - alloc_init_pud(pgd, addr, next, phys, prot, pgtable_alloc, + + /* + * If any intermediate levels are folded into the PGDs, we + * need to deal with the contiguous attributes here, since + * the contiguity can only be observed at this level. + */ + if (PGDIR_SHIFT == PMD_SHIFT && !page_mappings_only && + ((addr | phys) & ~CONT_PMD_MASK) == 0) { + if (end - addr >= CONT_PMD_SIZE) + __prot = __pgprot(pgprot_val(prot) | + PTE_CONT); + else + __prot = prot; + } else if (PGDIR_SHIFT == PUD_SHIFT && CONT_PUD_SHIFT > 0 && + !page_mappings_only && + ((addr | phys) & ~CONT_PUD_MASK) == 0) { + if (end - addr >= CONT_PUD_SIZE) + __prot = __pgprot(pgprot_val(prot) | + PTE_CONT); + else + __prot = prot; + } + alloc_init_pud(pgd, addr, next, phys, __prot, pgtable_alloc, page_mappings_only); phys += next - addr; } while (pgd++, addr = next, addr != end);