mbox series

[00/29] block/export: Add infrastructure and QAPI for block exports

Message ID 20200907182011.521007-1-kwolf@redhat.com
Headers show
Series block/export: Add infrastructure and QAPI for block exports | expand

Message

Kevin Wolf Sept. 7, 2020, 6:19 p.m. UTC
We are planning to add more block export types than just NBD in the near
future (e.g. vhost-user-blk and FUSE). This series lays the ground for
this with some generic block export infrastructure and QAPI interfaces
that will allow managing all of them (for now add/remove/query).

As a side effect, qemu-storage-daemon can now map --export directly to
the block-export-add QMP command, similar to other command line options.
The built-in NBD servers also gains new options that bring it at least a
little closer to feature parity with qemu-nbd.

Kevin Wolf (29):
  nbd: Remove unused nbd_export_get_blockdev()
  qapi: Create block-export module
  qapi: Rename BlockExport to BlockExportOptions
  block/export: Add BlockExport infrastructure and block-export-add
  qemu-storage-daemon: Use qmp_block_export_add()
  qemu-nbd: Use raw block driver for --offset
  block/export: Remove magic from block-export-add
  nbd: Add max-connections to nbd-server-start
  nbd: Add writethrough to block-export-add
  nbd: Remove NBDExport.close callback
  qemu-nbd: Use blk_exp_add() to create the export
  nbd/server: Simplify export shutdown
  block/export: Move refcount from NBDExport to BlockExport
  block/export: Move AioContext from NBDExport to BlockExport
  block/export: Add node-name to BlockExportOptions
  block/export: Allocate BlockExport in blk_exp_add()
  block/export: Add blk_exp_close_all(_type)
  block/export: Add 'id' option to block-export-add
  block/export: Move strong user reference to block_exports
  block/export: Add block-export-del
  block/export: Add BLOCK_EXPORT_DELETED event
  block/export: Move blk to BlockExport
  block/export: Create BlockBackend in blk_exp_add()
  block/export: Add query-block-exports
  block/export: Move writable to BlockExportOptions
  nbd: Merge nbd_export_new() and nbd_export_create()
  nbd: Deprecate nbd-server-add/remove
  iotests: Factor out qemu_tool_pipe_and_status()
  iotests: Test block-export-* QMP interface

 qapi/block-core.json                 | 166 --------------
 qapi/block-export.json               | 291 ++++++++++++++++++++++++
 qapi/qapi-schema.json                |   1 +
 docs/system/deprecated.rst           |   8 +
 include/block/export.h               |  89 ++++++++
 include/block/nbd.h                  |  22 +-
 block.c                              |   2 +-
 block/export/export.c                | 318 +++++++++++++++++++++++++++
 block/monitor/block-hmp-cmds.c       |  13 +-
 blockdev-nbd.c                       | 171 +++++++-------
 nbd/server.c                         | 309 +++++++++++---------------
 qemu-nbd.c                           |  67 +++---
 storage-daemon/qemu-storage-daemon.c |  27 +--
 tests/qemu-iotests/iotests.py        |  59 ++---
 block/export/meson.build             |   1 +
 block/meson.build                    |   2 +
 meson.build                          |   2 +-
 qapi/meson.build                     |   4 +-
 storage-daemon/qapi/qapi-schema.json |   1 +
 tests/qemu-iotests/140.out           |   1 +
 tests/qemu-iotests/223.out           |   8 +-
 tests/qemu-iotests/307               | 117 ++++++++++
 tests/qemu-iotests/307.out           | 111 ++++++++++
 tests/qemu-iotests/group             |   1 +
 24 files changed, 1267 insertions(+), 524 deletions(-)
 create mode 100644 qapi/block-export.json
 create mode 100644 include/block/export.h
 create mode 100644 block/export/export.c
 create mode 100644 block/export/meson.build
 create mode 100755 tests/qemu-iotests/307
 create mode 100644 tests/qemu-iotests/307.out

Comments

Max Reitz Sept. 10, 2020, 10:53 a.m. UTC | #1
On 07.09.20 20:19, Kevin Wolf wrote:
> nbd-server-add tries to be convenient and adds two questionable
> features that we don't want to share in block-export-add, even for NBD
> exports:
> 
> 1. When requesting a writable export of a read-only device, the export
>    is silently downgraded to read-only. This should be an error in the
>    context of block-export-add.
> 
> 2. When using a BlockBackend name, unplugging the device from the guest
>    will automatically stop the NBD server, too. This may sometimes be
>    what you want, but it could also be very surprising. Let's keep
>    things explicit with block-export-add. If the user wants to stop the
>    export, they should tell us so.
> 
> Move these things into the nbd-server-add QMP command handler so that
> they apply only there.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  include/block/export.h |  2 ++
>  include/block/nbd.h    |  3 ++-
>  block/export/export.c  | 13 +++++++++---
>  blockdev-nbd.c         | 47 +++++++++++++++++++++++++++++++++++-------
>  nbd/server.c           | 20 +++++++++++-------
>  qemu-nbd.c             |  3 +--
>  6 files changed, 67 insertions(+), 21 deletions(-)

[...]

> +    if (bdrv_is_read_only(bs)) {
> +        export_opts.u.nbd.has_writable = true;

Ah, yes, setting that might be nice. :)

Reviewed-by: Max Reitz <mreitz@redhat.com>
Max Reitz Sept. 10, 2020, 12:35 p.m. UTC | #2
On 07.09.20 20:19, Kevin Wolf wrote:
> Every block export needs a block node to export, so add a 'node-name'
> option to BlockExportOptions and remove the replaced option 'device'
> from BlockExportOptionsNbd.
> 
> To maintain compatibility in nbd-server-add, BlockExportOptionsNbd needs
> to be wrapped by a new type NbdServerAddOptions that adds 'device' back
> because nbd-server-add doesn't use the BlockExportOptions base type at
> all (so even without changing it to a 'node-name' option in
> block-export-add, this compatibility code would be necessary).
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  qapi/block-export.json         | 27 ++++++++++++++++-----
>  block/monitor/block-hmp-cmds.c |  6 ++---
>  blockdev-nbd.c                 | 44 +++++++++++++++++++++++++---------
>  qemu-nbd.c                     |  2 +-
>  4 files changed, 58 insertions(+), 21 deletions(-)

Reviewed-by: Max Reitz <mreitz@redhat.com>
Max Reitz Sept. 10, 2020, 1:22 p.m. UTC | #3
On 07.09.20 20:19, Kevin Wolf wrote:
> This adds a function to shut down all block exports, and another one to
> shut down the block exports of a single type. The latter is used for now
> when stopping the NBD server. As soon as we implement support for
> multiple NBD servers, we'll need a per-server list of exports and it
> will be replaced by a function using that.
> 
> As a side effect, the BlockExport layer has a list tracking all existing
> exports now. closed_exports loses its only user and can go away.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  include/block/export.h | 15 ++++++++
>  include/block/nbd.h    |  2 --
>  block.c                |  2 +-
>  block/export/export.c  | 79 ++++++++++++++++++++++++++++++++++++++++--
>  blockdev-nbd.c         |  2 +-
>  nbd/server.c           | 34 +++---------------
>  qemu-nbd.c             |  2 +-
>  7 files changed, 100 insertions(+), 36 deletions(-)

[...]

>  /* Callers must hold exp->ctx lock */
>  void blk_exp_unref(BlockExport *exp)
>  {
>      assert(exp->refcount > 0);
>      if (--exp->refcount == 0) {
> -        exp->drv->delete(exp);
> -        g_free(exp);
> +        /* Touch the block_exports list only in the main thread */
> +        aio_bh_schedule_oneshot(qemu_get_aio_context(), blk_exp_delete_bh,
> +                                exp);

Looks safe.

Reviewed-by: Max Reitz <mreitz@redhat.com>

(The effort of special-casing this to delete the export immediately if
we already run in the main thread doesn’t look worth it.)
Max Reitz Sept. 10, 2020, 1:26 p.m. UTC | #4
On 07.09.20 20:20, Kevin Wolf wrote:
> We'll need an id to identify block exports in monitor commands. This
> adds one.
> 
> Note that this is different from the 'name' option in the NBD server,
> which is the externally visible export name. While block export ids need
> to be unique in the whole process, export names must be unique only for
> the same server. Different export types or (potentially in the future)
> multiple NBD servers can have the same export name externally, but still
> need different block export ids internally.
> 
> Signed-off-by: Kevin Wolf <kwolf@redhat.com>
> ---
>  qapi/block-export.json               |  5 +++++
>  include/block/export.h               |  3 +++
>  block/export/export.c                | 26 ++++++++++++++++++++++++++
>  blockdev-nbd.c                       |  1 +
>  qemu-nbd.c                           |  1 +
>  storage-daemon/qemu-storage-daemon.c |  2 +-
>  tests/qemu-iotests/223.out           |  4 ++--
>  7 files changed, 39 insertions(+), 3 deletions(-)

Reviewed-by: Max Reitz <mreitz@redhat.com>