From patchwork Tue Apr 27 21:56:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Douglas Gilbert X-Patchwork-Id: 428295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 55EE2C433ED for ; Tue, 27 Apr 2021 21:57:47 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 30D16613B4 for ; Tue, 27 Apr 2021 21:57:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239033AbhD0V6a (ORCPT ); Tue, 27 Apr 2021 17:58:30 -0400 Received: from smtp.infotech.no ([82.134.31.41]:38698 "EHLO smtp.infotech.no" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235440AbhD0V63 (ORCPT ); Tue, 27 Apr 2021 17:58:29 -0400 Received: from localhost (localhost [127.0.0.1]) by smtp.infotech.no (Postfix) with ESMTP id 138E320426F; Tue, 27 Apr 2021 23:57:44 +0200 (CEST) X-Virus-Scanned: by amavisd-new-2.6.6 (20110518) (Debian) at infotech.no Received: from smtp.infotech.no ([127.0.0.1]) by localhost (smtp.infotech.no [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VpNMmLP6ARZ0; Tue, 27 Apr 2021 23:57:36 +0200 (CEST) Received: from xtwo70.bingwo.ca (host-45-58-219-4.dyn.295.ca [45.58.219.4]) by smtp.infotech.no (Postfix) with ESMTPA id 6FBBD20416A; Tue, 27 Apr 2021 23:57:34 +0200 (CEST) From: Douglas Gilbert To: linux-scsi@vger.kernel.org Cc: martin.petersen@oracle.com, jejb@linux.vnet.ibm.com, hare@suse.de Subject: [PATCH v18 00/83] sg: add v4 interface, request sharing Date: Tue, 27 Apr 2021 17:56:09 -0400 Message-Id: <20210427215733.417746-1-dgilbert@interlog.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org This is the combined patchset showing the additions in the second half patchset, that will be presented after the first half patchset is accepted. Patches 1 to 45 (inclusive) are a new version (v18) of the first half patchset and have their own cover letter whose subject line is: [PATCH v18 00/45] sg: add v4 interface The idea behind releasing the combined patchset is so they can be run through the kernel's code sanity checking mechanisms. So expect some noise. The additions in the second half patchset are more fully described in: https://sg.danny.cz/sg/sg_v40.html which is mirrored at: https://doug-gilbert.github.io/sg_v40.html The following list is a summary of features in the second patchset: - add (sg) file descriptor sharing; this is used by: - request sharing - multiple requests (mrq) use of SGV4_FLAG_DO_ON_OTHER flag - add request sharing, mainly to expedite copying. READ bio handed off to paired WRITE with no data to user space unless requested. Also capable of using VERIFY(BytChk=1) instead of the WRITE (what NVMe does in its Compare NVM command). - extend the request sharing logic so SGV4_FLAG_KEEP_SHARE will keep bio after a WRITE. This allows for a single source, multiple destinations copy - add an extensible SG_SET_GET_EXTENDED ioctl(2) that takes a fixed size structure (96 byte). - add multiple requests capability (mrq) in a single ioctl(SG_IO) or ioctl(SG_IOSUBMIT) invocation. Can be combined with request sharing. - add a SGV4_FLAG_IMMED flag for ioctl(SG_IORECEIVE) or ioctl(SG_IORECEIVE_V3) so they don't wait - add logic for (block layer generated) tag handling and keep existing pack_id (packet id) logic which plays a similar role - add ioctl(SG_IOABORT) to abort an inflight command/request using its pack-id or tag. - add shared variable blocking (svb) method to the mrq. Assumes it is doing copy-like request sharing. By default WRITEs are unordered (wrt to each other). With SGV4_FLAG_ORDERED_WR flag WRITEs are ordered as required for ZBC disks. - add support to pass a fd generated by eventfd(2) to the driver via an ioctl(2). - use iopoll/hipri/blk_poll with mrq, especially svb. - bump the driver version number to 4.0.47 Most of the above are _only_ implemented for the sg version 4 (i.e. based on struct sg_io_v4) interface. Douglas Gilbert (83): sg: move functions around sg: remove typedefs, type+formatting cleanup sg: sg_log and is_enabled sg: rework sg_poll(), minor changes sg: bitops in sg_device sg: make open count an atomic sg: move header to uapi section sg: speed sg_poll and sg_get_num_waiting sg: sg_allow_if_err_recovery and renames sg: improve naming sg: change rwlock to spinlock sg: ioctl handling sg: split sg_read sg: sg_common_write add structure for arguments sg: rework sg_vma_fault sg: rework sg_mmap sg: replace sg_allow_access sg: rework scatter gather handling sg: introduce request state machine sg: sg_find_srp_by_id sg: sg_fill_request_element sg: printk change %p to %pK sg: xarray for fds in device sg: xarray for reqs in fd sg: replace rq array with xarray sg: sense buffer rework sg: add sg v4 interface support sg: rework debug info sg: add 8 byte SCSI LUN to sg_scsi_id sg: expand sg_comm_wr_t sg: add sg_iosubmit_v3 and sg_ioreceive_v3 ioctls sg: add some __must_hold macros sg: move procfs objects to avoid forward decls sg: protect multiple receivers sg: first debugfs support sg: rework mmap support sg: defang allow_dio sg: warn v3 write system call users sg: add mmap_sz tracking sg: remove rcv_done request state sg: track lowest inactive and await indexes sg: remove unit attention check for device changed sg: no_dxfer: move to/from kernel buffers sg: add blk_poll support sg: bump version to 4.0.12 sg: add sg_ioabort ioctl sg: add sg_set_get_extended ioctl sg: sgat_elem_sz and sum_fd_dlens sg: tag and more_async sg: add fd sharing , change, unshare sg: add shared requests sg: add multiple request support sg: rename some mrq variables sg: unlikely likely sg: mrq abort sg: reduce atomic operations sg: add excl_wait flag sg: tweak sg_find_sfp_by_fd() sg: add snap_dev flag and snapped in debugfs sg: compress usercontext to uc sg: optionally output sg_request.frq_bm flags sg: work on sg_mrq_sanity() sg: shared variable blocking sg: device timestamp sg: condition met is not an error sg: split sg_setup_req sg: finish after read-side request sg: keep share and dout offset flags sg: add dlen to sg_comm_wr_t sg: make use of struct sg_mrq_hold sg: add mmap IO option for mrq metadata sg: add eventfd support sg: table of error number explanations sg: add ordered write flag sg: expand source line length to 100 characters sg: add no_attach_msg parameter sg: add SGV4_FLAG_REC_ORDER sg: max to read for mrq sg_ioreceive sg: mrq: if uniform svb then re-use bio_s sg: expand bvec usage; re-use bio_s sg: blk_poll/hipri work for mrq sg: pollable and non-pollable requests sg: bump version to 4.0.47 drivers/scsi/sg.c | 10003 +++++++++++++++++++++++++++++++-------- include/scsi/sg.h | 273 +- include/uapi/scsi/sg.h | 491 ++ 3 files changed, 8440 insertions(+), 2327 deletions(-) create mode 100644 include/uapi/scsi/sg.h