From patchwork Tue Aug 2 09:58:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Olivier Masse X-Patchwork-Id: 599965 Delivered-To: patches@linaro.org Received: by 2002:a05:7000:4388:0:0:0:0 with SMTP id w8csp290212mae; Thu, 25 Aug 2022 06:30:02 -0700 (PDT) X-Google-Smtp-Source: AA6agR4zBWzvHAePwQrECZnPz/mq26iBaAYfSuFHaNUW8qXEzo2fD4dbbsQgalqqQi+4a8UEGLEk X-Received: by 2002:a05:622a:1986:b0:343:225d:f9e1 with SMTP id u6-20020a05622a198600b00343225df9e1mr3525875qtc.651.1661434202028; Thu, 25 Aug 2022 06:30:02 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [3.208.193.21]) by mx.google.com with ESMTPS id jf10-20020a0562142a4a00b00492cce62509si10194834qvb.116.2022.08.25.06.30.01 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Aug 2022 06:30:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linaro-mm-sig-bounces+patches=linaro.org@lists.linaro.org designates 3.208.193.21 as permitted sender) client-ip=3.208.193.21; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@nxp.com header.s=selector2 header.b=hHeWLaNk; arc=fail (body hash mismatch); spf=pass (google.com: domain of linaro-mm-sig-bounces+patches=linaro.org@lists.linaro.org designates 3.208.193.21 as permitted sender) smtp.mailfrom="linaro-mm-sig-bounces+patches=linaro.org@lists.linaro.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=nxp.com Received: from lists.linaro.org (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id D5AA23F613 for ; Thu, 25 Aug 2022 13:30:01 +0000 (UTC) Received: from EUR05-DB8-obe.outbound.protection.outlook.com (mail-db8eur05on2062.outbound.protection.outlook.com [40.107.20.62]) by lists.linaro.org (Postfix) with ESMTPS id 00F704798B for ; Tue, 2 Aug 2022 09:58:53 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lhYAwWmzEbXWMWfplvvTPs2jdv7G5K/U4X06eCW2LDiUDLYFCrTWZNdvmwAbAKnFyV5r+T6SIqU1z+jZIXMnhiWPLKIUun4caev1LGsSmmYrybXBxybv95TxkHO74BZeensHNJyuH2TViLOP+Ra1LlMbVNQA9i4WJrJrrlFbXQuakvv7xkdwep04hl8RUjyGBK7khA17mdkiHS1wkj1nUR+NTMmhLJhD4E+8Dn5bEdsQnz9t8snIzVulVNvt2mHZilnpAn/Np6njvnLS0A5XaA+76Xq3gbhNfJXoQvtkZuF0p6Lid2ESWWJTLx+4WIsEC3+iHS18kZZvDnBvtMniKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=B6F4CMRdmn0GSb1NELVUv8fyXfsdW1uJtKJqqy0RZHs=; b=c04TM49Wfkm+Bng/rkCbN1wkDmskI1ZKdrsB8axBnq/NWm4ujX4jwrne9WSYqtJpJDuh05NDR6I79Cve8QnmigtG5rPjtkO9oxF+cvmE9jgILx7CpxpRVdpCyeYn5yD9uugBrPNVT3IxHCy2u7wXuJn0o5u5sg0Vo7axo/1awLun3/DG3Lfaonf7A0wA27AEIEn6zd1OsEe9pBWQlG0qs8LhnkP27aoShq+5c4b3aew0XN3a/nF5eUqseiAMP8gVq5lXqpEIf3jdFEdLNJrGMrinNyRbOdJ2e3R1tCnYtujEyusEKEpcU67h0ZDeuWt1ahId5v4HGI3jBI5akRfT0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=B6F4CMRdmn0GSb1NELVUv8fyXfsdW1uJtKJqqy0RZHs=; b=hHeWLaNkxVsyN7YIJ58EQ18YBZNAoOfutrJWRBbRDlwwq6dvopuX9ARso3HH86ArB9vC1zZpQRPXnwvZYQ5GOHRRnFC+MVnMdTy6izvE4lS6aEk8/5GOQXiZpIYOX1w7AuWleeuTccqAZlgr/a5KXjshZdKoDnNCUfEO/DwqZrk= Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nxp.com; Received: from AM0PR04MB6737.eurprd04.prod.outlook.com (2603:10a6:208:176::18) by AS8PR04MB8341.eurprd04.prod.outlook.com (2603:10a6:20b:3b0::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5482.16; Tue, 2 Aug 2022 09:58:52 +0000 Received: from AM0PR04MB6737.eurprd04.prod.outlook.com ([fe80::6ca7:299e:d92f:7d1b]) by AM0PR04MB6737.eurprd04.prod.outlook.com ([fe80::6ca7:299e:d92f:7d1b%5]) with mapi id 15.20.5482.011; Tue, 2 Aug 2022 09:58:52 +0000 From: Olivier Masse To: sumit.semwal@linaro.org, benjamin.gaignard@collabora.com, Brian.Starkey@arm.com, christian.koenig@amd.com, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, linux-kernel@vger.kernel.org Date: Tue, 2 Aug 2022 11:58:40 +0200 Message-Id: <20220802095843.14614-3-olivier.masse@nxp.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20220802095843.14614-1-olivier.masse@nxp.com> References: <20220802095843.14614-1-olivier.masse@nxp.com> X-ClientProxiedBy: AM3PR07CA0145.eurprd07.prod.outlook.com (2603:10a6:207:8::31) To AM0PR04MB6737.eurprd04.prod.outlook.com (2603:10a6:208:176::18) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d1aa91d2-bdc0-4ee8-19a8-08da746d9a88 X-MS-TrafficTypeDiagnostic: AS8PR04MB8341:EE_ X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OrZ+Mf2Im1BWeUgPbN/aeFSeGEekynp4KPB7T98fHlEIdvs8OvGsFdFjRALozCFA/gkdbPPx8avo2CD8IiVLwq+FeSb8bhmN6/XlBai9zjnk3AxtOwM3OLZlimRwL8CqtaFhuNRNKwlIDrMs5/4KhN5msNAX3uN5jV0zNOy6QCN1PzcY8iXmqTAeysEiahk7f5aUCnErBUBN5RLp/d0ErRHmNVel8fVNnN0Fvbiwvy+mSpV/px610mP07GPKuBuuD5mt9DmAW3e5bLCvWBVXhjYMvBhf+eLO4bjMq7RBV0T7wyvKk/p5jBheFx6tyC0FcFtx8GDT8MMnGpzvS1D5mzy50PZWhJmMpbkior99F0gtjL18MSveLPzWY2hKg6IVpXNCpmJb5xy6OogG1nHO4S0jD137ouRxhNPXsgdJEQQr6mJFOfEMOyNVhYh3E4YBWxajhnaDygzA/+sSWe91gaiB6b8Jp3hUrYaNTrlJ367MK7c3+OizXmgzbkSZjJAYX68XVnh8pG2cLmfaqW9W1zzXaI1lvhOYfZJ1PFxo99yupkGA7FV0SYfHOggsBOl/ep6QVvRzdsl7KftFJjK7HSsO6WoofMu5qv/GyPD9jv16t2B8eqrv4bnPg7tQ8wHkV35oDV3IoVqrkDBYHFFRkQYjCb0dczyEwr0rXbYgNOkiFdiSzIGHAC0L9t/RS0agtyy0VDFvRPCizmcZDeJng2mEoG22k4Mqv+2WKUuoKuf15iZnnxaKifyFewTFJNeJGUgUipQyIoS3Fbd0xOXrhKV9n4OiQUltOV11TZx6Crk= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AM0PR04MB6737.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230016)(4636009)(396003)(346002)(366004)(136003)(39860400002)(376002)(41300700001)(2906002)(83380400001)(2616005)(86362001)(6666004)(52116002)(6506007)(44832011)(1076003)(30864003)(36756003)(26005)(6512007)(38100700002)(186003)(38350700002)(6486002)(5660300002)(8676002)(66556008)(66476007)(66946007)(8936002)(478600001)(4326008)(316002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?q?9GrWezgtjpF/neKQaeu2sOFhzACQ?= =?utf-8?q?CpNpyXw5AT75hwFad626OyvF5/bjPp5X6wgI9da3SW4pplKq2oO3EC2uZqoJJvo6+?= =?utf-8?q?eDr/4ErDUWP1cyrhN6fan2+2Vq8AaDefj6+okul6nkGFiRG1xW+mO4xf4XAKg+FcZ?= =?utf-8?q?7v8KZN2ij1t4yO5Jrg5QIVtH4exOSpo07Mn+vw65Gq4eZ6hDYczaTx16UXhTj7Yap?= =?utf-8?q?MD0K4vWEi4hUgM54VAkdPDxaBZzsqWqN4FfM6KWKs6WH7R3pvwcwcph3EkBUhfzOi?= =?utf-8?q?EcXvj0aTMC9XOLRNZSEFfHu3rx9CvkxFSELrV7jiYisK47M8EoJkgqQ9jxsdoZ2bn?= =?utf-8?q?gtGVLr7vpo45C/K1GPPzcM7mSkyck+lhVgKCQ6yp0zzZEsF4SPAAJxwY74cpruwP8?= =?utf-8?q?n/U7jSzx6NYT5HRoeNgkw/HikNGFDMqQR7iMURu24ID4z9RFMg87Iweur/DCG1xrE?= =?utf-8?q?o+Vs/BQoK/pKbPuPt7902ruINKN3gDWMz6LAwOdyH1F9wBmsbeAXadQ1FPg7Zfrvg?= =?utf-8?q?Db+sWG3m+dZg/+ywZOcs0PCjAZTt8Zigqsvze0/gwAJfClJtlaJk2NZa7bphXv1Qo?= =?utf-8?q?8WaZHNs2trpMprc3yOG5KwdEGmVjZ7JOr/J1eIY+KVicHnL4wRAjVrsF6L30n9/Jh?= =?utf-8?q?kBHV0z8BcHPFoSLeiTznINlnJiQnhqXGnAqLSoeD8dbmuKJXFb3/6HP8DMs0z8j9j?= =?utf-8?q?KTPRoFjw5vgF9a+lu9x7K4JQpuufu2HAiY7+xql35tr8ZSWkOGg0zw/wJ5Cw7MBKk?= =?utf-8?q?vb4dpaEuuadUFU8bcX0fTdH0hKutAIQULh409eltO6wWYFlgw08V+9OqMv7LYci6I?= =?utf-8?q?az5DcFLbibpquFxrxJcjP+ELI6x9CV58hJIxs0FY5MglfeLHzVinwNSD/UipjqYp1?= =?utf-8?q?EhOv/eVJw7RTLcCUGKnFFmuHCwDg9qzyb+7FVWPTSDXm469l6H6zlrdxUwTjgx8yE?= =?utf-8?q?jRfCEDDwIQQtdD9UTntn5l+SuU/Jb9NsD635oMqo0UzDy7JXRybfBloTZt81W408K?= =?utf-8?q?Xa94DgczyElh+wcs3hdk6IXV22GV1MMLQIDm6GpVFfJvu5aCAO5u82UYf3zagUMz2?= =?utf-8?q?Q11qdiqvTLnhbXCfkQoLM9dDkrEZQWyCsznAG2q5G0iI3lpnsUMt85ED3wocYtSZz?= =?utf-8?q?7kbg1IKy141QlFXp9yDOqCtcmo8fvkcySzg+pQcG1XyiaAZ1VAhlMTUZs6+iY36v0?= =?utf-8?q?BFh8inzs0T3oS7bUbkdA1JFWHbNajQJdHkJE003G+WX3asytLtSALTt79Ei/KY6Ne?= =?utf-8?q?A94UXrcabdh1gw5sKURhECXZ0/FnXhUC8Sw3vVHfiqqMoxjCuO0HCTiO5kRbB2dlE?= =?utf-8?q?U19GHk1N68Cy4iccCAcryEZFBl1HA9VkZsg5pYk5snv3C9w2bnP5OXIG+SEPDoP/g?= =?utf-8?q?wD//f6RUATYymq1frrTtdNGeu8GtfM36X7sceuH62BoiHU9cmHwi+SzlbaV2XhQyH?= =?utf-8?q?y03V0uP1xpWj0o7rigEyUf0OtWjNgYeIJ99deCwwGqn1Qp+rdTfbqzUe9BkSK6iQ3?= =?utf-8?q?2qnk95Y7Wilb?= X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-Network-Message-Id: d1aa91d2-bdc0-4ee8-19a8-08da746d9a88 X-MS-Exchange-CrossTenant-AuthSource: AM0PR04MB6737.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Aug 2022 09:58:52.0620 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JJVQHhZVNHPI+vhQST2F/FbSJjoy0Oqz6BAmLzShGIaCRPcNiniHit7RCDbtHPsZK50coI1r7PjNQ7KlBC0mfw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB8341 X-MailFrom: olivier.masse@nxp.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation Message-ID-Hash: RJVE5ZOFA5X5PWENRU5V2SEFJFCR35TF X-Message-ID-Hash: RJVE5ZOFA5X5PWENRU5V2SEFJFCR35TF X-Mailman-Approved-At: Thu, 25 Aug 2022 13:29:17 +0000 CC: clement.faure@nxp.com, olivier.masse@nxp.com X-Mailman-Version: 3.3.5 Precedence: list Subject: [Linaro-mm-sig] [PATCH 2/5] ANDROID: dma-buf: heaps: Add a shrinker controlled page pool List-Id: "Unified memory management interest group." Archived-At: List-Archive: List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: From: John Stultz This patch adds a simple shrinker controlled page pool to the dmabuf heaps subsystem. This replaces the use of the networking page_pool, over concerns that the lack of a shrinker for that implementation may cause additional low-memory kills TODO: Take another pass at trying to unify this w/ the ttm pool Thoughts and feedback would be greatly appreciated! Cc: Sumit Semwal Cc: Liam Mark Cc: Laura Abbott Cc: Brian Starkey Cc: Hridya Valsaraju Cc: Suren Baghdasaryan Cc: Sandeep Patil Cc: Daniel Mentz Cc: Chris Goldsworthy Cc: Ørjan Eide Cc: Robin Murphy Cc: Ezequiel Garcia Cc: Simon Ser Cc: James Jones Cc: Andrew Morton Cc: Dave Hansen Cc: linux-mm@kvack.org Cc: linux-media@vger.kernel.org Cc: dri-devel@lists.freedesktop.org Signed-off-by: John Stultz Signed-off-by: Olivier Masse Bug: 168742043 --- drivers/dma-buf/heaps/Kconfig | 3 + drivers/dma-buf/heaps/Makefile | 1 + drivers/dma-buf/heaps/page_pool.c | 246 ++++++++++++++++++++++++++++++ drivers/dma-buf/heaps/page_pool.h | 55 +++++++ 4 files changed, 305 insertions(+) create mode 100644 drivers/dma-buf/heaps/page_pool.c create mode 100644 drivers/dma-buf/heaps/page_pool.h diff --git a/drivers/dma-buf/heaps/Kconfig b/drivers/dma-buf/heaps/Kconfig index 8ee64277a5d2..6a33193a7b3e 100644 --- a/drivers/dma-buf/heaps/Kconfig +++ b/drivers/dma-buf/heaps/Kconfig @@ -1,6 +1,9 @@ config DMABUF_HEAPS_DEFERRED_FREE tristate +config DMABUF_HEAPS_PAGE_POOL + tristate + config DMABUF_HEAPS_SYSTEM bool "DMA-BUF System Heap" depends on DMABUF_HEAPS diff --git a/drivers/dma-buf/heaps/Makefile b/drivers/dma-buf/heaps/Makefile index 5de95b77e169..e70722ea615e 100644 --- a/drivers/dma-buf/heaps/Makefile +++ b/drivers/dma-buf/heaps/Makefile @@ -1,5 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DMABUF_HEAPS_DEFERRED_FREE) += deferred-free-helper.o +obj-$(CONFIG_DMABUF_HEAPS_PAGE_POOL) += page_pool.o obj-$(CONFIG_DMABUF_HEAPS_SYSTEM) += system_heap.o obj-$(CONFIG_DMABUF_HEAPS_CMA) += cma_heap.o obj-$(CONFIG_DMABUF_HEAPS_DSP) += dsp_heap.o diff --git a/drivers/dma-buf/heaps/page_pool.c b/drivers/dma-buf/heaps/page_pool.c new file mode 100644 index 000000000000..3dd4c3862dca --- /dev/null +++ b/drivers/dma-buf/heaps/page_pool.c @@ -0,0 +1,246 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * DMA BUF page pool system + * + * Copyright (C) 2020 Linaro Ltd. + * + * Based on the ION page pool code + * Copyright (C) 2011 Google, Inc. + */ + +#include +#include +#include +#include +#include +#include "page_pool.h" + +static LIST_HEAD(pool_list); +static DEFINE_MUTEX(pool_list_lock); + +static struct page *dmabuf_page_pool_alloc_pages(struct dmabuf_page_pool *pool) +{ + if (fatal_signal_pending(current)) + return NULL; + return alloc_pages(pool->gfp_mask, pool->order); +} + +static void dmabuf_page_pool_free_pages(struct dmabuf_page_pool *pool, + struct page *page) +{ + __free_pages(page, pool->order); +} + +static void dmabuf_page_pool_add(struct dmabuf_page_pool *pool, struct page *page) +{ + int index; + + if (PageHighMem(page)) + index = POOL_HIGHPAGE; + else + index = POOL_LOWPAGE; + + mutex_lock(&pool->mutex); + list_add_tail(&page->lru, &pool->items[index]); + pool->count[index]++; + mod_node_page_state(page_pgdat(page), NR_KERNEL_MISC_RECLAIMABLE, + 1 << pool->order); + mutex_unlock(&pool->mutex); +} + +static struct page *dmabuf_page_pool_remove(struct dmabuf_page_pool *pool, int index) +{ + struct page *page; + + mutex_lock(&pool->mutex); + page = list_first_entry_or_null(&pool->items[index], struct page, lru); + if (page) { + pool->count[index]--; + list_del(&page->lru); + mod_node_page_state(page_pgdat(page), NR_KERNEL_MISC_RECLAIMABLE, + -(1 << pool->order)); + } + mutex_unlock(&pool->mutex); + + return page; +} + +static struct page *dmabuf_page_pool_fetch(struct dmabuf_page_pool *pool) +{ + struct page *page = NULL; + + page = dmabuf_page_pool_remove(pool, POOL_HIGHPAGE); + if (!page) + page = dmabuf_page_pool_remove(pool, POOL_LOWPAGE); + + return page; +} + +struct page *dmabuf_page_pool_alloc(struct dmabuf_page_pool *pool) +{ + struct page *page = NULL; + + if (WARN_ON(!pool)) + return NULL; + + page = dmabuf_page_pool_fetch(pool); + if (!page) + page = dmabuf_page_pool_alloc_pages(pool); + + return page; +} +EXPORT_SYMBOL_GPL(dmabuf_page_pool_alloc); + +void dmabuf_page_pool_free(struct dmabuf_page_pool *pool, struct page *page) +{ + if (WARN_ON(pool->order != compound_order(page))) + return; + + dmabuf_page_pool_add(pool, page); +} +EXPORT_SYMBOL_GPL(dmabuf_page_pool_free); + +static int dmabuf_page_pool_total(struct dmabuf_page_pool *pool, bool high) +{ + int count = pool->count[POOL_LOWPAGE]; + + if (high) + count += pool->count[POOL_HIGHPAGE]; + + return count << pool->order; +} + +struct dmabuf_page_pool *dmabuf_page_pool_create(gfp_t gfp_mask, unsigned int order) +{ + struct dmabuf_page_pool *pool = kmalloc(sizeof(*pool), GFP_KERNEL); + int i; + + if (!pool) + return NULL; + + for (i = 0; i < POOL_TYPE_SIZE; i++) { + pool->count[i] = 0; + INIT_LIST_HEAD(&pool->items[i]); + } + pool->gfp_mask = gfp_mask | __GFP_COMP; + pool->order = order; + mutex_init(&pool->mutex); + + mutex_lock(&pool_list_lock); + list_add(&pool->list, &pool_list); + mutex_unlock(&pool_list_lock); + + return pool; +} +EXPORT_SYMBOL_GPL(dmabuf_page_pool_create); + +void dmabuf_page_pool_destroy(struct dmabuf_page_pool *pool) +{ + struct page *page; + int i; + + /* Remove us from the pool list */ + mutex_lock(&pool_list_lock); + list_del(&pool->list); + mutex_unlock(&pool_list_lock); + + /* Free any remaining pages in the pool */ + for (i = 0; i < POOL_TYPE_SIZE; i++) { + while ((page = dmabuf_page_pool_remove(pool, i))) + dmabuf_page_pool_free_pages(pool, page); + } + + kfree(pool); +} +EXPORT_SYMBOL_GPL(dmabuf_page_pool_destroy); + +static int dmabuf_page_pool_do_shrink(struct dmabuf_page_pool *pool, gfp_t gfp_mask, + int nr_to_scan) +{ + int freed = 0; + bool high; + + if (current_is_kswapd()) + high = true; + else + high = !!(gfp_mask & __GFP_HIGHMEM); + + if (nr_to_scan == 0) + return dmabuf_page_pool_total(pool, high); + + while (freed < nr_to_scan) { + struct page *page; + + /* Try to free low pages first */ + page = dmabuf_page_pool_remove(pool, POOL_LOWPAGE); + if (!page) + page = dmabuf_page_pool_remove(pool, POOL_HIGHPAGE); + + if (!page) + break; + + dmabuf_page_pool_free_pages(pool, page); + freed += (1 << pool->order); + } + + return freed; +} + +static int dmabuf_page_pool_shrink(gfp_t gfp_mask, int nr_to_scan) +{ + struct dmabuf_page_pool *pool; + int nr_total = 0; + int nr_freed; + bool only_scan = false; + + if (!nr_to_scan) + only_scan = true; + + mutex_lock(&pool_list_lock); + list_for_each_entry(pool, &pool_list, list) { + if (only_scan) { + nr_total += dmabuf_page_pool_do_shrink(pool, + gfp_mask, + nr_to_scan); + } else { + nr_freed = dmabuf_page_pool_do_shrink(pool, + gfp_mask, + nr_to_scan); + nr_to_scan -= nr_freed; + nr_total += nr_freed; + if (nr_to_scan <= 0) + break; + } + } + mutex_unlock(&pool_list_lock); + + return nr_total; +} + +static unsigned long dmabuf_page_pool_shrink_count(struct shrinker *shrinker, + struct shrink_control *sc) +{ + return dmabuf_page_pool_shrink(sc->gfp_mask, 0); +} + +static unsigned long dmabuf_page_pool_shrink_scan(struct shrinker *shrinker, + struct shrink_control *sc) +{ + if (sc->nr_to_scan == 0) + return 0; + return dmabuf_page_pool_shrink(sc->gfp_mask, sc->nr_to_scan); +} + +struct shrinker pool_shrinker = { + .count_objects = dmabuf_page_pool_shrink_count, + .scan_objects = dmabuf_page_pool_shrink_scan, + .seeks = DEFAULT_SEEKS, + .batch = 0, +}; + +static int dmabuf_page_pool_init_shrinker(void) +{ + return register_shrinker(&pool_shrinker); +} +module_init(dmabuf_page_pool_init_shrinker); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/dma-buf/heaps/page_pool.h b/drivers/dma-buf/heaps/page_pool.h new file mode 100644 index 000000000000..e3ec9eaacbc2 --- /dev/null +++ b/drivers/dma-buf/heaps/page_pool.h @@ -0,0 +1,55 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * DMA BUF PagePool implementation + * Based on earlier ION code by Google + * + * Copyright (C) 2011 Google, Inc. + * Copyright (C) 2020 Linaro Ltd. + */ + +#ifndef _DMABUF_PAGE_POOL_H +#define _DMABUF_PAGE_POOL_H + +#include +#include +#include +#include +#include +#include + +/* page types we track in the pool */ +enum { + POOL_LOWPAGE, /* Clean lowmem pages */ + POOL_HIGHPAGE, /* Clean highmem pages */ + + POOL_TYPE_SIZE +}; + +/** + * struct dmabuf_page_pool - pagepool struct + * @count[]: array of number of pages of that type in the pool + * @items[]: array of list of pages of the specific type + * @mutex: lock protecting this struct and especially the count + * item list + * @gfp_mask: gfp_mask to use from alloc + * @order: order of pages in the pool + * @list: list node for list of pools + * + * Allows you to keep a pool of pre allocated pages to use + */ +struct dmabuf_page_pool { + int count[POOL_TYPE_SIZE]; + struct list_head items[POOL_TYPE_SIZE]; + struct mutex mutex; + gfp_t gfp_mask; + unsigned int order; + struct list_head list; +}; + +struct dmabuf_page_pool *dmabuf_page_pool_create(gfp_t gfp_mask, + unsigned int order); +void dmabuf_page_pool_destroy(struct dmabuf_page_pool *pool); +struct page *dmabuf_page_pool_alloc(struct dmabuf_page_pool *pool); +void dmabuf_page_pool_free(struct dmabuf_page_pool *pool, struct page *page); + +#endif /* _DMABUF_PAGE_POOL_H */