From patchwork Tue Apr 3 14:10:17 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marek Szyprowski X-Patchwork-Id: 7597 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 7C236199213 for ; Tue, 3 Apr 2012 14:10:57 +0000 (UTC) Received: from mail-iy0-f180.google.com (mail-iy0-f180.google.com [209.85.210.180]) by fiordland.canonical.com (Postfix) with ESMTP id 43829A1855D for ; Tue, 3 Apr 2012 14:10:57 +0000 (UTC) Received: by mail-iy0-f180.google.com with SMTP id e36so8036573iag.11 for ; Tue, 03 Apr 2012 07:10:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:date:from :in-reply-to:to:message-id:mime-version:x-mailer:references:cc :subject:x-beenthere:x-mailman-version:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :content-type:content-transfer-encoding:sender:errors-to :x-gm-message-state; bh=lILSVO/8AllSvuzsPrsBlZYEBRYfYDu243n4HSmazkA=; b=KTYDBMSPYpy9wju8zCo0EUYFfw06GMGG2cUuTBmA9yUBbgXNwQNoDTpWqnU13Dhn1t twqOH7pqqxZ3Es/aUUvliNmQkn/OmIJvnFjL8sIBRf6JigD1vNdVyaPdBMADc/2yqvuy 3EcacrzxEbYmM0t33PFEdmpdL3YpfYN7DRvXuajAt9iYG4tNbiq/HPWwa2juWjwamSTB JuDR5wL7y0b3zue4QF5nrzVmFPMMT+DtvkoNZphUCEFyMZs/G7uzlxNbJoQbebPzXJsC uopgMs9lvBMwkeQognkvp0eD0yQAR/tRvqL07b4Eo9YtUAdwY+PFY5vEoYmoSCCJIbyq C8ug== Received: by 10.50.187.231 with SMTP id fv7mr1940816igc.51.1333462256803; Tue, 03 Apr 2012 07:10:56 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.5.205 with SMTP id 13csp34830ibw; Tue, 3 Apr 2012 07:10:56 -0700 (PDT) Received: by 10.180.92.71 with SMTP id ck7mr36717467wib.21.1333462250238; Tue, 03 Apr 2012 07:10:50 -0700 (PDT) Received: from mombin.canonical.com (mombin.canonical.com. [91.189.95.16]) by mx.google.com with ESMTP id cf5si15114363wib.22.2012.04.03.07.10.49; Tue, 03 Apr 2012 07:10:50 -0700 (PDT) Received-SPF: neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) client-ip=91.189.95.16; Authentication-Results: mx.google.com; spf=neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) smtp.mail=linaro-mm-sig-bounces@lists.linaro.org Received: from localhost ([127.0.0.1] helo=mombin.canonical.com) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1SF4Ra-0004J1-Tq; Tue, 03 Apr 2012 14:10:47 +0000 Received: from mailout2.w1.samsung.com ([210.118.77.12]) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1SF4RM-00048m-K3 for linaro-mm-sig@lists.linaro.org; Tue, 03 Apr 2012 14:10:32 +0000 Received: from euspt2 (mailout2.w1.samsung.com [210.118.77.12]) by mailout2.w1.samsung.com (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTP id <0M1W004SHQ1915@mailout2.w1.samsung.com> for linaro-mm-sig@lists.linaro.org; Tue, 03 Apr 2012 15:10:29 +0100 (BST) Received: from linux.samsung.com ([106.116.38.10]) by spt2.w1.samsung.com (iPlanet Messaging Server 5.2 Patch 2 (built Jul 14 2004)) with ESMTPA id <0M1W00DU0Q1A2J@spt2.w1.samsung.com> for linaro-mm-sig@lists.linaro.org; Tue, 03 Apr 2012 15:10:27 +0100 (BST) Received: from mcdsrvbld02.digital.local (unknown [106.116.37.23]) by linux.samsung.com (Postfix) with ESMTP id B2D6B27006E; Tue, 03 Apr 2012 16:16:21 +0200 (CEST) Date: Tue, 03 Apr 2012 16:10:17 +0200 From: Marek Szyprowski In-reply-to: <1333462221-3987-1-git-send-email-m.szyprowski@samsung.com> To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-media@vger.kernel.org, linux-mm@kvack.org, linaro-mm-sig@lists.linaro.org Message-id: <1333462221-3987-13-git-send-email-m.szyprowski@samsung.com> MIME-version: 1.0 X-Mailer: git-send-email 1.7.9.1 References: <1333462221-3987-1-git-send-email-m.szyprowski@samsung.com> Cc: Ohad Ben-Cohen , Daniel Walker , Russell King , Arnd Bergmann , Jonathan Corbet , Mel Gorman , Michal Nazarewicz , Dave Hansen , Jesse Barker , Kyungmin Park , Andrew Morton , Rob Clark , KAMEZAWA Hiroyuki Subject: [Linaro-mm-sig] [PATCHv24 12/16] mm: trigger page reclaim in alloc_contig_range() to stabilise watermarks X-BeenThere: linaro-mm-sig@lists.linaro.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Unified memory management interest group." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linaro-mm-sig-bounces@lists.linaro.org Errors-To: linaro-mm-sig-bounces@lists.linaro.org X-Gm-Message-State: ALoCoQl+f/Ys7ltoYkZ0KG8lt0zZ9qoiu0pxbXwUjHsGZt/nCtoJILqIQI3urPnMvB6deEQ5BXSG alloc_contig_range() performs memory allocation so it also should keep track on keeping the correct level of memory watermarks. This commit adds a call to *_slowpath style reclaim to grab enough pages to make sure that the final collection of contiguous pages from freelists will not starve the system. Signed-off-by: Marek Szyprowski Signed-off-by: Kyungmin Park CC: Michal Nazarewicz Tested-by: Rob Clark Tested-by: Ohad Ben-Cohen Tested-by: Benjamin Gaignard Tested-by: Robert Nelson Tested-by: Barry Song --- include/linux/mmzone.h | 9 +++++++ mm/page_alloc.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+), 0 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 8c1335f..26f2040 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -63,8 +63,10 @@ enum { #ifdef CONFIG_CMA # define is_migrate_cma(migratetype) unlikely((migratetype) == MIGRATE_CMA) +# define cma_wmark_pages(zone) zone->min_cma_pages #else # define is_migrate_cma(migratetype) false +# define cma_wmark_pages(zone) 0 #endif #define for_each_migratetype_order(order, type) \ @@ -371,6 +373,13 @@ struct zone { /* see spanned/present_pages for more description */ seqlock_t span_seqlock; #endif +#ifdef CONFIG_CMA + /* + * CMA needs to increase watermark levels during the allocation + * process to make sure that the system is not starved. + */ + unsigned long min_cma_pages; +#endif struct free_area free_area[MAX_ORDER]; #ifndef CONFIG_SPARSEMEM diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5a90ada..17a5521 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5079,6 +5079,11 @@ static void __setup_per_zone_wmarks(void) zone->watermark[WMARK_LOW] = min_wmark_pages(zone) + (tmp >> 2); zone->watermark[WMARK_HIGH] = min_wmark_pages(zone) + (tmp >> 1); + + zone->watermark[WMARK_MIN] += cma_wmark_pages(zone); + zone->watermark[WMARK_LOW] += cma_wmark_pages(zone); + zone->watermark[WMARK_HIGH] += cma_wmark_pages(zone); + setup_zone_migrate_reserve(zone); spin_unlock_irqrestore(&zone->lock, flags); } @@ -5684,6 +5689,54 @@ static int __alloc_contig_migrate_range(unsigned long start, unsigned long end) return ret > 0 ? 0 : ret; } +/* + * Update zone's cma pages counter used for watermark level calculation. + */ +static inline void __update_cma_watermarks(struct zone *zone, int count) +{ + unsigned long flags; + spin_lock_irqsave(&zone->lock, flags); + zone->min_cma_pages += count; + spin_unlock_irqrestore(&zone->lock, flags); + setup_per_zone_wmarks(); +} + +/* + * Trigger memory pressure bump to reclaim some pages in order to be able to + * allocate 'count' pages in single page units. Does similar work as + *__alloc_pages_slowpath() function. + */ +static int __reclaim_pages(struct zone *zone, gfp_t gfp_mask, int count) +{ + enum zone_type high_zoneidx = gfp_zone(gfp_mask); + struct zonelist *zonelist = node_zonelist(0, gfp_mask); + int did_some_progress = 0; + int order = 1; + + /* + * Increase level of watermarks to force kswapd do his job + * to stabilise at new watermark level. + */ + __update_cma_watermarks(zone, count); + + /* Obey watermarks as if the page was being allocated */ + while (!zone_watermark_ok(zone, 0, low_wmark_pages(zone), 0, 0)) { + wake_all_kswapd(order, zonelist, high_zoneidx, zone_idx(zone)); + + did_some_progress = __perform_reclaim(gfp_mask, order, zonelist, + NULL); + if (!did_some_progress) { + /* Exhausted what can be done so it's blamo time */ + out_of_memory(zonelist, gfp_mask, order, NULL, false); + } + } + + /* Restore original watermark levels. */ + __update_cma_watermarks(zone, -count); + + return count; +} + /** * alloc_contig_range() -- tries to allocate given range of pages * @start: start PFN to allocate @@ -5782,6 +5835,13 @@ int alloc_contig_range(unsigned long start, unsigned long end, goto done; } + /* + * Reclaim enough pages to make sure that contiguous allocation + * will not starve the system. + */ + __reclaim_pages(zone, GFP_HIGHUSER_MOVABLE, end-start); + + /* Grab isolated pages from freelists. */ outer_end = isolate_freepages_range(outer_start, end); if (!outer_end) { ret = -EBUSY;