[25/27] afs: Make {Y,}FS.FetchData an asynchronous operation

Message ID	20241024140539.3828093-26-dhowells@redhat.com
State	Superseded
Headers	show Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2C67220CCCB for <ceph-devel@vger.kernel.org>; Thu, 24 Oct 2024 14:09:09 +0000 (UTC) From: David Howells <dhowells@redhat.com> To: Christian Brauner <christian@brauner.io>, Steve French <smfrench@gmail.com>, Matthew Wilcox <willy@infradead.org> Cc: David Howells <dhowells@redhat.com>, Jeff Layton <jlayton@kernel.org>, Gao Xiang <hsiangkao@linux.alibaba.com>, Dominique Martinet <asmadeus@codewreck.org>, Marc Dionne <marc.dionne@auristor.com>, Paulo Alcantara <pc@manguebit.com>, Shyam Prasad N <sprasad@microsoft.com>, Tom Talpey <tom@talpey.com>, Eric Van Hensbergen <ericvh@kernel.org>, Ilya Dryomov <idryomov@gmail.com>, netfs@lists.linux.dev, linux-afs@lists.infradead.org, linux-cifs@vger.kernel.org, linux-nfs@vger.kernel.org, ceph-devel@vger.kernel.org, v9fs@lists.linux.dev, linux-erofs@lists.ozlabs.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 25/27] afs: Make {Y,}FS.FetchData an asynchronous operation Date: Thu, 24 Oct 2024 15:05:23 +0100 Message-ID: <20241024140539.3828093-26-dhowells@redhat.com> In-Reply-To: <20241024140539.3828093-1-dhowells@redhat.com> References: <20241024140539.3828093-1-dhowells@redhat.com> Precedence: bulk MIME-Version: 1.0 Content-Transfer-Encoding: 8bit
Series	netfs: Read performance improvements and "single-blob" support \| expand [00/27] netfs: Read performance improvements and "single-blob" support [01/27] netfs: Remove call to folio_index() [02/27] netfs: Fix a few minor bugs in netfs_page_mkwrite() [03/27] netfs: Remove unnecessary references to pages [04/27] netfs: Use a folio_queue allocation and free functions [05/27] netfs: Add a tracepoint to log the lifespan of folio_queue structs [06/27] netfs: Abstract out a rolling folio buffer implementation [07/27] netfs: Make netfs_advance_write() return size_t [08/27] netfs: Split retry code out of fs/netfs/write_collect.c [09/27] netfs: Drop the error arg from netfs_read_subreq_terminated() [10/27] netfs: Drop the was_async arg from netfs_read_subreq_terminated() [11/27] netfs: Don't use bh spinlock [12/27] afs: Don't use mutex for I/O operation lock [13/27] afs: Fix EEXIST error returned from afs_rmdir() to be ENOTEMPTY [14/27] afs: Fix directory format encoding struct [15/27] netfs: Remove some extraneous directory invalidations [16/27] cachefiles: Add some subrequest tracepoints [17/27] cachefiles: Add auxiliary data trace [18/27] afs: Add more tracepoints to do with tracking validity [19/27] netfs: Add functions to build/clean a buffer in a folio_queue [20/27] netfs: Add support for caching single monolithic objects such as AFS dirs [21/27] afs: Make afs_init_request() get a key if not given a file [22/27] afs: Use netfslib for directories [23/27] afs: Use netfslib for symlinks, allowing them to be cached [24/27] afs: Eliminate afs_read [25/27] afs: Make {Y,}FS.FetchData an asynchronous operation [26/27] netfs: Change the read result collector to only use one work item [27/27] afs: Make afs_mkdir() locally initialise a new directory's content

diff --git a/fs/afs/file.c b/fs/afs/file.c index b996f4419c0c..6c1ec3b2411d 100644 --- a/fs/afs/file.c +++ b/fs/afs/file.c @@ -239,12 +239,91 @@ const struct afs_operation_ops afs_fetch_data_operation = { .put = afs_fetch_data_put, }; +static void afs_issue_read_call(struct afs_operation *op) +{ + op->call_responded = false; + op->call_error = 0; + op->call_abort_code = 0; + if (test_bit(AFS_SERVER_FL_IS_YFS, &op->server->flags)) + yfs_fs_fetch_data(op); + else + afs_fs_fetch_data(op); +} + +static void afs_end_read(struct afs_operation *op) +{ + if (op->call_responded && op->server) + set_bit(AFS_SERVER_FL_RESPONDING, &op->server->flags); + + if (!afs_op_error(op)) + afs_fetch_data_success(op); + else if (op->cumul_error.aborted) + afs_fetch_data_aborted(op); + else + afs_fetch_data_notify(op); + + afs_end_vnode_operation(op); + afs_put_operation(op); +} + +/* + * Perform I/O processing on an asynchronous call. The work item carries a ref + * to the call struct that we either need to release or to pass on. + */ +static void afs_read_receive(struct afs_call *call) +{ + struct afs_operation *op = call->op; + enum afs_call_state state; + + _enter(""); + + state = READ_ONCE(call->state); + if (state == AFS_CALL_COMPLETE) + return; + + while (state < AFS_CALL_COMPLETE && READ_ONCE(call->need_attention)) { + WRITE_ONCE(call->need_attention, false); + afs_deliver_to_call(call); + state = READ_ONCE(call->state); + } + + if (state < AFS_CALL_COMPLETE) { + netfs_read_subreq_progress(op->fetch.subreq); + if (rxrpc_kernel_check_life(call->net->socket, call->rxcall)) + return; + /* rxrpc terminated the call. */ + afs_set_call_complete(call, call->error, call->abort_code); + } + + op->call_abort_code = call->abort_code; + op->call_error = call->error; + op->call_responded = call->responded; + afs_put_call(op->call); + + /* If the call failed, then we need to crank the server rotation + * handle and try the next. + */ + if (afs_select_fileserver(op)) { + afs_issue_read_call(op); + return; + } + + afs_end_read(op); +} + +void afs_fetch_data_async_rx(struct work_struct *work) +{ + struct afs_call *call = container_of(work, struct afs_call, async_work); + + afs_read_receive(call); + afs_put_call(call); +} + /* * Fetch file data from the volume. */ -static void afs_read_worker(struct work_struct *work) +static void afs_issue_read(struct netfs_io_subrequest *subreq) { - struct netfs_io_subrequest *subreq = container_of(work, struct netfs_io_subrequest, work); struct afs_operation *op; struct afs_vnode *vnode = AFS_FS_I(subreq->rreq->inode); struct key *key = subreq->rreq->netfs_priv; @@ -269,13 +348,27 @@ static void afs_read_worker(struct work_struct *work) op->ops = &afs_fetch_data_operation; trace_netfs_sreq(subreq, netfs_sreq_trace_submit); - afs_do_sync_operation(op); -} -static void afs_issue_read(struct netfs_io_subrequest *subreq) -{ - INIT_WORK(&subreq->work, afs_read_worker); - queue_work(system_long_wq, &subreq->work); + if (subreq->rreq->origin == NETFS_READAHEAD || + subreq->rreq->iocb) { + op->flags |= AFS_OPERATION_ASYNC; + + if (!afs_begin_vnode_operation(op)) { + subreq->error = PTR_ERR(op); + netfs_read_subreq_terminated(subreq); + afs_put_operation(op); + return; + } + + if (!afs_select_fileserver(op)) { + afs_end_read(op); + return; + } + + afs_issue_read_call(op); + } else { + afs_do_sync_operation(op); + } } static int afs_init_request(struct netfs_io_request *rreq, struct file *file) diff --git a/fs/afs/fs_operation.c b/fs/afs/fs_operation.c index 8488ff8183fa..0b1338d65ae6 100644 --- a/fs/afs/fs_operation.c +++ b/fs/afs/fs_operation.c @@ -256,7 +256,7 @@ bool afs_begin_vnode_operation(struct afs_operation *op) /* * Tidy up a filesystem cursor and unlock the vnode. */ -static void afs_end_vnode_operation(struct afs_operation *op) +void afs_end_vnode_operation(struct afs_operation *op) { _enter(""); diff --git a/fs/afs/fsclient.c b/fs/afs/fsclient.c index d9d224c95454..6380cdcfd4fc 100644 --- a/fs/afs/fsclient.c +++ b/fs/afs/fsclient.c @@ -352,7 +352,6 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call) ret = afs_extract_data(call, true); subreq->transferred += count_before - call->iov_len; call->remaining -= count_before - call->iov_len; - netfs_read_subreq_progress(subreq); if (ret < 0) return ret; @@ -409,6 +408,7 @@ static int afs_deliver_fs_fetch_data(struct afs_call *call) static const struct afs_call_type afs_RXFSFetchData = { .name = "FS.FetchData", .op = afs_FS_FetchData, + .async_rx = afs_fetch_data_async_rx, .deliver = afs_deliver_fs_fetch_data, .destructor = afs_flat_call_destructor, }; @@ -416,6 +416,7 @@ static const struct afs_call_type afs_RXFSFetchData = { static const struct afs_call_type afs_RXFSFetchData64 = { .name = "FS.FetchData64", .op = afs_FS_FetchData64, + .async_rx = afs_fetch_data_async_rx, .deliver = afs_deliver_fs_fetch_data, .destructor = afs_flat_call_destructor, }; @@ -436,6 +437,9 @@ static void afs_fs_fetch_data64(struct afs_operation *op) if (!call) return afs_op_nomem(op); + if (op->flags & AFS_OPERATION_ASYNC) + call->async = true; + /* marshall the parameters */ bp = call->request; bp[0] = htonl(FSFETCHDATA64); diff --git a/fs/afs/internal.h b/fs/afs/internal.h index 36125fce0590..c7f0d75eab7f 100644 --- a/fs/afs/internal.h +++ b/fs/afs/internal.h @@ -202,6 +202,9 @@ struct afs_call_type { /* clean up a call */ void (*destructor)(struct afs_call *call); + /* Async receive processing function */ + void (*async_rx)(struct work_struct *work); + /* Work function */ void (*work)(struct work_struct *work); @@ -941,6 +944,7 @@ struct afs_operation { #define AFS_OPERATION_TRIED_ALL 0x0400 /* Set if we've tried all the fileservers */ #define AFS_OPERATION_RETRY_SERVER 0x0800 /* Set if we should retry the current server */ #define AFS_OPERATION_DIR_CONFLICT 0x1000 /* Set if we detected a 3rd-party dir change */ +#define AFS_OPERATION_ASYNC 0x2000 /* Set if should run asynchronously */ }; /* @@ -1103,6 +1107,7 @@ extern int afs_cache_wb_key(struct afs_vnode *, struct afs_file *); extern void afs_put_wb_key(struct afs_wb_key *); extern int afs_open(struct inode *, struct file *); extern int afs_release(struct inode *, struct file *); +void afs_fetch_data_async_rx(struct work_struct *work); /* * flock.c @@ -1154,6 +1159,7 @@ extern void afs_fs_store_acl(struct afs_operation *); extern struct afs_operation *afs_alloc_operation(struct key *, struct afs_volume *); extern int afs_put_operation(struct afs_operation *); extern bool afs_begin_vnode_operation(struct afs_operation *); +extern void afs_end_vnode_operation(struct afs_operation *op); extern void afs_wait_for_operation(struct afs_operation *); extern int afs_do_sync_operation(struct afs_operation *); @@ -1325,6 +1331,7 @@ extern void afs_charge_preallocation(struct work_struct *); extern void afs_put_call(struct afs_call *); void afs_deferred_put_call(struct afs_call *call); void afs_make_call(struct afs_call *call, gfp_t gfp); +void afs_deliver_to_call(struct afs_call *call); void afs_wait_for_call_to_complete(struct afs_call *call); extern struct afs_call *afs_alloc_flat_call(struct afs_net *, const struct afs_call_type *, diff --git a/fs/afs/main.c b/fs/afs/main.c index a14f6013e316..1ae0067f772d 100644 --- a/fs/afs/main.c +++ b/fs/afs/main.c @@ -177,7 +177,7 @@ static int __init afs_init(void) afs_wq = alloc_workqueue("afs", 0, 0); if (!afs_wq) goto error_afs_wq; - afs_async_calls = alloc_workqueue("kafsd", WQ_MEM_RECLAIM, 0); + afs_async_calls = alloc_workqueue("kafsd", WQ_MEM_RECLAIM | WQ_UNBOUND, 0); if (!afs_async_calls) goto error_async; afs_lock_manager = alloc_workqueue("kafs_lockd", WQ_MEM_RECLAIM, 0); diff --git a/fs/afs/rxrpc.c b/fs/afs/rxrpc.c index 9f2a3bb56ec6..94fff4e214b0 100644 --- a/fs/afs/rxrpc.c +++ b/fs/afs/rxrpc.c @@ -149,7 +149,8 @@ static struct afs_call *afs_alloc_call(struct afs_net *net, call->net = net; call->debug_id = atomic_inc_return(&rxrpc_debug_id); refcount_set(&call->ref, 1); - INIT_WORK(&call->async_work, afs_process_async_call); + INIT_WORK(&call->async_work, type->async_rx ?: afs_process_async_call); + INIT_WORK(&call->work, call->type->work); INIT_WORK(&call->free_work, afs_deferred_free_worker); init_waitqueue_head(&call->waitq); spin_lock_init(&call->state_lock); @@ -254,8 +255,6 @@ static struct afs_call *afs_get_call(struct afs_call *call, static void afs_queue_call_work(struct afs_call *call) { if (call->type->work) { - INIT_WORK(&call->work, call->type->work); - afs_get_call(call, afs_call_trace_work); if (!queue_work(afs_wq, &call->work)) afs_put_call(call); @@ -501,7 +500,7 @@ static void afs_log_error(struct afs_call *call, s32 remote_abort) /* * deliver messages to a call */ -static void afs_deliver_to_call(struct afs_call *call) +void afs_deliver_to_call(struct afs_call *call) { enum afs_call_state state; size_t len; @@ -803,6 +802,7 @@ static int afs_deliver_cm_op_id(struct afs_call *call) return -ENOTSUPP; trace_afs_cb_call(call); + call->work.func = call->type->work; /* pass responsibility for the remainer of this message off to the * cache manager op */ diff --git a/fs/afs/write.c b/fs/afs/write.c index 17d188aaf101..e87b55792aa8 100644 --- a/fs/afs/write.c +++ b/fs/afs/write.c @@ -193,6 +193,18 @@ void afs_retry_request(struct netfs_io_request *wreq, struct netfs_io_stream *st list_first_entry(&stream->subrequests, struct netfs_io_subrequest, rreq_link); + switch (wreq->origin) { + case NETFS_READAHEAD: + case NETFS_READPAGE: + case NETFS_READ_GAPS: + case NETFS_READ_SINGLE: + case NETFS_READ_FOR_WRITE: + case NETFS_DIO_READ: + return; + default: + break; + } + switch (subreq->error) { case -EACCES: case -EPERM: diff --git a/fs/afs/yfsclient.c b/fs/afs/yfsclient.c index 3718d852fabc..4e7d93ee5a08 100644 --- a/fs/afs/yfsclient.c +++ b/fs/afs/yfsclient.c @@ -397,7 +397,6 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call) ret = afs_extract_data(call, true); subreq->transferred += count_before - call->iov_len; - netfs_read_subreq_progress(subreq); if (ret < 0) return ret; @@ -457,6 +456,7 @@ static int yfs_deliver_fs_fetch_data64(struct afs_call *call) static const struct afs_call_type yfs_RXYFSFetchData64 = { .name = "YFS.FetchData64", .op = yfs_FS_FetchData64, + .async_rx = afs_fetch_data_async_rx, .deliver = yfs_deliver_fs_fetch_data64, .destructor = afs_flat_call_destructor, }; @@ -486,6 +486,9 @@ void yfs_fs_fetch_data(struct afs_operation *op) if (!call) return afs_op_nomem(op); + if (op->flags & AFS_OPERATION_ASYNC) + call->async = true; + /* marshall the parameters */ bp = call->request; bp = xdr_encode_u32(bp, YFSFETCHDATA64);

[25/27] afs: Make {Y,}FS.FetchData an asynchronous operation

Commit Message

Patch