ceph: try to choose the auth MDS if possible for getattr

If any 'x' caps is issued we can just choose the auth MDS instead
of the random replica MDSes. Because only when the Locker is in
LOCK_EXEC state will the loner client could get the 'x' caps. And
if we send the getattr requests to any replica MDS it must auth pin
and tries to rdlock from the auth MDS, and then the auth MDS need
to do the Locker state transition to LOCK_SYNC. And after that the
lock state will change back.

This cost much when doing the Locker state transition and usually
will need to revoke caps from clients.

URL: https://tracker.ceph.com/issues/55240
Signed-off-by: Xiubo Li <xiubli@redhat.com>
---
 fs/ceph/inode.c | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

Will send a V2 to fix the same issue in ceph_netfs_issue_op_inline().

-- Xiubo


On 4/21/22 12:20 PM, Xiubo Li wrote:
> If any 'x' caps is issued we can just choose the auth MDS instead
> of the random replica MDSes. Because only when the Locker is in
> LOCK_EXEC state will the loner client could get the 'x' caps. And
> if we send the getattr requests to any replica MDS it must auth pin
> and tries to rdlock from the auth MDS, and then the auth MDS need
> to do the Locker state transition to LOCK_SYNC. And after that the
> lock state will change back.
>
> This cost much when doing the Locker state transition and usually
> will need to revoke caps from clients.
>
> URL: https://tracker.ceph.com/issues/55240
> Signed-off-by: Xiubo Li <xiubli@redhat.com>
> ---
>   fs/ceph/inode.c | 20 +++++++++++++++++++-
>   1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/fs/ceph/inode.c b/fs/ceph/inode.c
> index b45f321910af..2a5023b1272d 100644
> --- a/fs/ceph/inode.c
> +++ b/fs/ceph/inode.c
> @@ -2270,6 +2270,7 @@ int __ceph_do_getattr(struct inode *inode, struct page *locked_page,
>   	struct ceph_fs_client *fsc = ceph_sb_to_client(inode->i_sb);
>   	struct ceph_mds_client *mdsc = fsc->mdsc;
>   	struct ceph_mds_request *req;
> +	int issued = ceph_caps_issued(ceph_inode(inode));
>   	int mode;
>   	int err;
>   
> @@ -2283,7 +2284,24 @@ int __ceph_do_getattr(struct inode *inode, struct page *locked_page,
>   	if (!force && ceph_caps_issued_mask_metric(ceph_inode(inode), mask, 1))
>   			return 0;
>   
> -	mode = (mask & CEPH_STAT_RSTAT) ? USE_AUTH_MDS : USE_ANY_MDS;
> +	/*
> +	 * If any 'x' caps is issued we can just choose the auth MDS
> +	 * instead of the random replica MDSes. Because only when the
> +	 * Locker is in LOCK_EXEC state will the exclusive client could
> +	 * get the 'x' caps. And if we send the getattr requests to any
> +	 * replica MDS it must auth pin and tries to rdlock from the auth
> +	 * MDS, and then the auth MDS need to do the Locker state transition
> +	 * to LOCK_SYNC. And after that the lock state will change back.
> +	 *
> +	 * This cost much when doing the Locker state transition and
> +	 * usually will need to revoke caps from clients.
> +	 */
> +	if (((mask & CEPH_CAP_ANY_SHARED) && (issued & CEPH_CAP_ANY_EXCL))
> +	    || (mask & CEPH_STAT_RSTAT))
> +		mode = USE_AUTH_MDS;
> +	else
> +		mode = USE_ANY_MDS;
> +
>   	req = ceph_mdsc_create_request(mdsc, CEPH_MDS_OP_GETATTR, mode);
>   	if (IS_ERR(req))
>   		return PTR_ERR(req);