From patchwork Wed Jun 13 01:11:00 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 9241 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 2788223EB4 for ; Wed, 13 Jun 2012 01:11:18 +0000 (UTC) Received: from mail-gg0-f180.google.com (mail-gg0-f180.google.com [209.85.161.180]) by fiordland.canonical.com (Postfix) with ESMTP id D5DB3A1818F for ; Wed, 13 Jun 2012 01:11:17 +0000 (UTC) Received: by ggnf1 with SMTP id f1so29194ggn.11 for ; Tue, 12 Jun 2012 18:11:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-forwarded-to:x-forwarded-for:delivered-to:received-spf:from:to:cc :subject:date:message-id:x-mailer:in-reply-to:references :x-content-scanned:x-cbid:x-gm-message-state; bh=09z15WqhgjFnUL8qygFN9QBUzGMxxrF5kLBqWxLzwgU=; b=LkDBCMric5WG70A/3Axgx6EIwPa5T7l0kfUykOthOOP5pFy7I3k1Mzf2EoA4jHU/iL Mf6Rs1zPBqBrAxF2bA/81vtD+ZnocL+cyZMFvVfppM8u2rorJpL+zs3A7s5dHy/tPs5I k9VE1ARHoFkZcdSKVWeQD8lBO6w/qJGzrPn/KPop5iKCTzvsbQO/PVNI8Cy0r5uLl/Hp x40TJ+CS7rePc2iG9HwQ85493WhmIxy39mfrXPowe2mDM/8wMym34vCpjD3w4E1nCqLb Cdo4MCFomE8fTF+KDyzKsrRf/2r6wwUla01etYUrQ+Pbnu+kNX8lKo07eK8ANXcMgiXp qIAA== Received: by 10.50.203.39 with SMTP id kn7mr9560769igc.53.1339549876936; Tue, 12 Jun 2012 18:11:16 -0700 (PDT) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.231.24.148 with SMTP id v20csp198982ibb; Tue, 12 Jun 2012 18:11:15 -0700 (PDT) Received: by 10.236.190.6 with SMTP id d6mr30086720yhn.16.1339549874581; Tue, 12 Jun 2012 18:11:14 -0700 (PDT) Received: from e2.ny.us.ibm.com (e2.ny.us.ibm.com. [32.97.182.142]) by mx.google.com with ESMTPS id j46si1496379yhm.128.2012.06.12.18.11.14 (version=TLSv1/SSLv3 cipher=OTHER); Tue, 12 Jun 2012 18:11:14 -0700 (PDT) Received-SPF: pass (google.com: domain of jstultz@us.ibm.com designates 32.97.182.142 as permitted sender) client-ip=32.97.182.142; Authentication-Results: mx.google.com; spf=pass (google.com: domain of jstultz@us.ibm.com designates 32.97.182.142 as permitted sender) smtp.mail=jstultz@us.ibm.com Received: from /spool/local by e2.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Tue, 12 Jun 2012 21:11:13 -0400 Received: from d01dlp01.pok.ibm.com (9.56.224.56) by e2.ny.us.ibm.com (192.168.1.102) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Tue, 12 Jun 2012 21:11:11 -0400 Received: from d01relay07.pok.ibm.com (d01relay07.pok.ibm.com [9.56.227.147]) by d01dlp01.pok.ibm.com (Postfix) with ESMTP id 46A2138C8054; Tue, 12 Jun 2012 21:11:11 -0400 (EDT) Received: from d01av04.pok.ibm.com (d01av04.pok.ibm.com [9.56.224.64]) by d01relay07.pok.ibm.com (8.13.8/8.13.8/NCO v10.0) with ESMTP id q5D1BAAi35520710; Tue, 12 Jun 2012 21:11:10 -0400 Received: from d01av04.pok.ibm.com (loopback [127.0.0.1]) by d01av04.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVout) with ESMTP id q5D1B8hW011400; Tue, 12 Jun 2012 21:11:10 -0400 Received: from kernel.beaverton.ibm.com (kernel.beaverton.ibm.com [9.47.67.96]) by d01av04.pok.ibm.com (8.14.4/8.13.1/NCO v10.0 AVin) with ESMTP id q5D1B8Y0011375; Tue, 12 Jun 2012 21:11:08 -0400 Received: by kernel.beaverton.ibm.com (Postfix, from userid 1056) id 9C921C0626; Tue, 12 Jun 2012 18:11:06 -0700 (PDT) From: John Stultz To: LKML Cc: John Stultz , Andrew Morton , Android Kernel Team , Robert Love , Mel Gorman , Hugh Dickins , Dave Hansen , Rik van Riel , Dmitry Adamushko , Dave Chinner , Neil Brown , Andrea Righi , "Aneesh Kumar K.V" , Taras Glek , Mike Hommey , Jan Kara , KOSAKI Motohiro Subject: [PATCH 5/6] [RFC][HACK] tmpfs: Purge volatile ranges on writepage instead of using shrinker Date: Tue, 12 Jun 2012 18:11:00 -0700 Message-Id: <1339549862-653-6-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 1.7.3.2.146.gca209 In-Reply-To: <1339549862-653-1-git-send-email-john.stultz@linaro.org> References: <1339549862-653-1-git-send-email-john.stultz@linaro.org> X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12061301-5112-0000-0000-000008F6591D X-Gm-Message-State: ALoCoQnlX2XuuBrb7l69Ao9yya10wRZJZtoG37Mp15MWLLPZSuPswWtA58L5x7CaBJeLDphOZWdT In order to avoid using a shrinker, purge volatile ranges on writepage. This requires deactivating/activating the page ranges together to ensure relative LRU behavior in purging the volatile ranges. This no longer requires the volatile range lru so remove that code. Also add volatile range infrastructure to find a range that contains a given page. NOTE: On systems without swap, the VM won't shrink anonymous memory, so writepage is never called and volatile ranges won't be purged. This issue will be addressed in a following patch. I've only been able to do minimal testing, so there's probably much still wrong with this patch. But hopefully it will provide a concrete approach for discussion. CC: Andrew Morton CC: Android Kernel Team CC: Robert Love CC: Mel Gorman CC: Hugh Dickins CC: Dave Hansen CC: Rik van Riel CC: Dmitry Adamushko CC: Dave Chinner CC: Neil Brown CC: Andrea Righi CC: Aneesh Kumar K.V CC: Taras Glek CC: Mike Hommey CC: Jan Kara CC: KOSAKI Motohiro Signed-off-by: John Stultz --- include/linux/volatile.h | 11 ++---- mm/shmem.c | 90 ++++++++++++++++++++------------------------- mm/volatile.c | 84 +++++++++++-------------------------------- 3 files changed, 64 insertions(+), 121 deletions(-) diff --git a/include/linux/volatile.h b/include/linux/volatile.h index 51f3d6e..8edb5c1 100644 --- a/include/linux/volatile.h +++ b/include/linux/volatile.h @@ -5,15 +5,11 @@ struct volatile_fs_head { struct mutex lock; - struct list_head lru_head; - s64 unpurged_page_count; }; #define DEFINE_VOLATILE_FS_HEAD(name) struct volatile_fs_head name = { \ .lock = __MUTEX_INITIALIZER(name.lock), \ - .lru_head = LIST_HEAD_INIT(name.lru_head), \ - .unpurged_page_count = 0, \ } @@ -34,12 +30,11 @@ extern long volatile_range_remove(struct volatile_fs_head *head, struct address_space *mapping, pgoff_t start_index, pgoff_t end_index); -extern s64 volatile_range_lru_size(struct volatile_fs_head *head); - extern void volatile_range_clear(struct volatile_fs_head *head, struct address_space *mapping); -extern s64 volatile_ranges_pluck_lru(struct volatile_fs_head *head, - struct address_space **mapping, +int volatile_page_in_range(struct volatile_fs_head *head, + struct page *page, loff_t *start, loff_t *end); + #endif /* _LINUX_VOLATILE_H */ diff --git a/mm/shmem.c b/mm/shmem.c index 0513816..8dbb8ac 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -629,7 +629,7 @@ static DEFINE_VOLATILE_FS_HEAD(shmem_volatile_head); static int shmem_mark_volatile(struct inode *inode, loff_t offset, loff_t len) { - loff_t lstart, lend; + loff_t lstart, lend, index; int ret; lstart = offset >> PAGE_CACHE_SHIFT; @@ -643,6 +643,17 @@ static int shmem_mark_volatile(struct inode *inode, loff_t offset, loff_t len) ((lend+1)<> PAGE_CACHE_SHIFT; @@ -660,6 +671,18 @@ static int shmem_unmark_volatile(struct inode *inode, loff_t offset, loff_t len) ret = volatile_range_remove(&shmem_volatile_head, &inode->i_data, lstart, lend); + + for (index = lstart; index < lend; index++) { + struct page *page; + int error = shmem_getpage(inode, index, &page, SGP_FALLOC,NULL); + if (error) + break; + + activate_page(page); + unlock_page(page); + page_cache_release(page); + } + volatile_range_unlock(&shmem_volatile_head); return ret; @@ -672,54 +695,6 @@ static void shmem_clear_volatile(struct inode *inode) volatile_range_unlock(&shmem_volatile_head); } -static -int shmem_volatile_shrink(struct shrinker *ignored, struct shrink_control *sc) -{ - s64 nr_to_scan = sc->nr_to_scan; - const gfp_t gfp_mask = sc->gfp_mask; - struct address_space *mapping; - loff_t start, end; - int ret; - s64 page_count; - - if (nr_to_scan && !(gfp_mask & __GFP_FS)) - return -1; - - volatile_range_lock(&shmem_volatile_head); - page_count = volatile_range_lru_size(&shmem_volatile_head); - if (!nr_to_scan) - goto out; - - do { - ret = volatile_ranges_pluck_lru(&shmem_volatile_head, - &mapping, &start, &end); - if (ret) { - shmem_truncate_range(mapping->host, - start< 0)); - -out: - volatile_range_unlock(&shmem_volatile_head); - - return page_count; -} - -static struct shrinker shmem_volatile_shrinker = { - .shrink = shmem_volatile_shrink, - .seeks = DEFAULT_SEEKS, -}; - -static int __init shmem_shrinker_init(void) -{ - register_shrinker(&shmem_volatile_shrinker); - return 0; -} -arch_initcall(shmem_shrinker_init); - static void shmem_evict_inode(struct inode *inode) { @@ -884,14 +859,29 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc) struct inode *inode; swp_entry_t swap; pgoff_t index; + loff_t start, end; BUG_ON(!PageLocked(page)); mapping = page->mapping; index = page->index; inode = mapping->host; info = SHMEM_I(inode); + if (info->flags & VM_LOCKED) goto redirty; + + volatile_range_lock(&shmem_volatile_head); + if (volatile_page_in_range(&shmem_volatile_head, page, &start, &end)) { + unlock_page(page); + volatile_range_unlock(&shmem_volatile_head); + shmem_truncate_range(inode, start<interval_node.end - range->interval_node.start; - new_size = end_index-start_index; - - if (!range->purged) - head->unpurged_page_count += new_size - old_size; - range->interval_node.start = start_index; range->interval_node.end = end_index; } @@ -185,11 +177,6 @@ static void vrange_del(struct volatile_fs_head *head, struct interval_tree_root *root, struct volatile_range *vrange) { - if (!vrange->purged) { - head->unpurged_page_count -= - vrange->interval_node.end - vrange->interval_node.start; - list_del(&vrange->lru); - } interval_tree_remove(root, &vrange->interval_node); kfree(vrange); } @@ -309,11 +296,6 @@ long volatile_range_add(struct volatile_fs_head *head, new->purged = purged; interval_tree_add(root, &new->interval_node); - /* Only add unpurged ranges to LRU */ - if (!purged) { - head->unpurged_page_count += end - start; - list_add_tail(&new->lru, &head->lru_head); - } return purged; } @@ -400,8 +382,6 @@ long volatile_range_remove(struct volatile_fs_head *head, new->purged = vrange->purged; vrange_resize(head, vrange, node->start, start-1); interval_tree_add(root, &new->interval_node); - if (!new->purged) - list_add_tail(&new->lru, &head->lru_head); break; } @@ -414,62 +394,40 @@ out: return ret; } -/** - * volatile_range_lru_size: Returns the number of unpurged pages on the lru - * @head: per-fs volatile head - * - * Returns the number of unpurged pages on the LRU - * - * Must lock the volatile_fs_head before calling! - * - */ -s64 volatile_range_lru_size(struct volatile_fs_head *head) -{ - WARN_ON(!mutex_is_locked(&head->lock)); - return head->unpurged_page_count; -} - -/** - * volatile_ranges_pluck_lru: Returns mapping and size of lru unpurged range - * @head: per-fs volatile head - * @mapping: dbl pointer to mapping who's range is being purged - * @start: Pointer to starting address of range being purged - * @end: Pointer to ending address of range being purged - * - * Returns the mapping, start and end values of the least recently used - * range. Marks the range as purged and removes it from the LRU. - * - * Must lock the volatile_fs_head before calling! - * - * Returns 1 on success if a range was returned - * Return 0 if no ranges were found. - */ -s64 volatile_ranges_pluck_lru(struct volatile_fs_head *head, - struct address_space **mapping, +int volatile_page_in_range(struct volatile_fs_head *head, + struct page *page, loff_t *start, loff_t *end) { - struct volatile_range *range; + struct interval_tree_root *root; + struct interval_tree_node *node; + int ret = 0; WARN_ON(!mutex_is_locked(&head->lock)); - if (list_empty(&head->lru_head)) + root = mapping_to_root(page->mapping); + if (!root) return 0; - range = list_first_entry(&head->lru_head, struct volatile_range, lru); - - *start = range->interval_node.start; - *end = range->interval_node.end; - *mapping = range->mapping; + node = interval_tree_in_interval(root, page->index, page->index); + if (node) { + struct volatile_range *vrange; - head->unpurged_page_count -= *end - *start; - list_del(&range->lru); - range->purged = 1; + vrange = container_of(node, struct volatile_range, + interval_node); - return 1; + if (!vrange->purged) { + *start = vrange->interval_node.start; + *end = vrange->interval_node.end; + vrange->purged = 1; + ret = 1; + } + } + return ret; } + /* * Cleans up any volatile ranges. */