mbox series

[PULL,v2,00/30] Block patches

Message ID 20201012182800.157697-1-stefanha@redhat.com
Headers show
Series Block patches | expand

Message

Stefan Hajnoczi Oct. 12, 2020, 6:27 p.m. UTC
The following changes since commit 2387df497b4b4bcf754eb7398edca82889e2ef54:

  Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' into staging (2020-10-12 11:29:42 +0100)

are available in the Git repository at:

  https://gitlab.com/stefanha/qemu.git tags/block-pull-request

for you to fetch changes up to 3664ec6bbe236126b79d251d4037889e7181ab55:

  iotests: add commit top->base cases to 274 (2020-10-12 16:47:58 +0100)

----------------------------------------------------------------
Pull request

v2:
 * Rebase and resolve conflict with commit 029a88c9a7e3 ("qemu-nbd: Honor
   SIGINT and SIGHUP") [Peter]

----------------------------------------------------------------

Coiby Xu (7):
  libvhost-user: Allow vu_message_read to be replaced
  libvhost-user: remove watch for kick_fd when de-initialize vu-dev
  util/vhost-user-server: generic vhost user server
  block: move logical block size check function to a common utility
    function
  block/export: vhost-user block device backend server
  test: new qTest case to test the vhost-user-blk-server
  MAINTAINERS: Add vhost-user block device backend server maintainer

Philippe Mathieu-Daudé (1):
  block/nvme: Add driver statistics for access alignment and hw errors

Stefan Hajnoczi (17):
  util/vhost-user-server: s/fileds/fields/ typo fix
  util/vhost-user-server: drop unnecessary QOM cast
  util/vhost-user-server: drop unnecessary watch deletion
  block/export: consolidate request structs into VuBlockReq
  util/vhost-user-server: drop unused DevicePanicNotifier
  util/vhost-user-server: fix memory leak in vu_message_read()
  util/vhost-user-server: check EOF when reading payload
  util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
  block/export: report flush errors
  block/export: convert vhost-user-blk server to block export API
  util/vhost-user-server: move header to include/
  util/vhost-user-server: use static library in meson.build
  qemu-storage-daemon: avoid compiling blockdev_ss twice
  block: move block exports to libblockdev
  block/export: add iothread and fixed-iothread options
  block/export: add vhost-user-blk multi-queue support
  tests/qtest: add multi-queue test case to vhost-user-blk-test

Vladimir Sementsov-Ogievskiy (5):
  block/io: fix bdrv_co_block_status_above
  block/io: bdrv_common_block_status_above: support include_base
  block/io: bdrv_common_block_status_above: support bs == base
  block/io: fix bdrv_is_allocated_above
  iotests: add commit top->base cases to 274

 MAINTAINERS                                |  10 +
 qapi/block-core.json                       |  24 +-
 qapi/block-export.json                     |  36 +-
 block/coroutines.h                         |   2 +
 block/export/vhost-user-blk-server.h       |  19 +
 contrib/libvhost-user/libvhost-user.h      |  21 +
 include/qemu/vhost-user-server.h           |  65 ++
 tests/qtest/libqos/libqtest.h              |  17 +
 tests/qtest/libqos/vhost-user-blk.h        |  48 ++
 util/block-helpers.h                       |  19 +
 block/export/export.c                      |  37 +-
 block/export/vhost-user-blk-server.c       | 431 +++++++++++
 block/io.c                                 | 132 ++--
 block/nvme.c                               |  27 +
 block/qcow2.c                              |  16 +-
 contrib/libvhost-user/libvhost-user-glib.c |   2 +-
 contrib/libvhost-user/libvhost-user.c      |  15 +-
 hw/core/qdev-properties-system.c           |  31 +-
 nbd/server.c                               |   2 -
 qemu-nbd.c                                 |  25 +-
 softmmu/vl.c                               |   4 +
 stubs/blk-exp-close-all.c                  |   7 +
 tests/qtest/libqos/vhost-user-blk.c        | 129 ++++
 tests/qtest/libqtest.c                     |  36 +-
 tests/qtest/vhost-user-blk-test.c          | 822 +++++++++++++++++++++
 tests/vhost-user-bridge.c                  |   2 +
 tools/virtiofsd/fuse_virtio.c              |   4 +-
 util/block-helpers.c                       |  46 ++
 util/vhost-user-server.c                   | 446 +++++++++++
 block/export/meson.build                   |   3 +-
 contrib/libvhost-user/meson.build          |   1 +
 meson.build                                |  22 +-
 nbd/meson.build                            |   2 +
 storage-daemon/meson.build                 |   3 +-
 stubs/meson.build                          |   1 +
 tests/qemu-iotests/274                     |  20 +
 tests/qemu-iotests/274.out                 |  68 ++
 tests/qtest/libqos/meson.build             |   1 +
 tests/qtest/meson.build                    |   4 +-
 util/meson.build                           |   4 +
 40 files changed, 2476 insertions(+), 128 deletions(-)
 create mode 100644 block/export/vhost-user-blk-server.h
 create mode 100644 include/qemu/vhost-user-server.h
 create mode 100644 tests/qtest/libqos/vhost-user-blk.h
 create mode 100644 util/block-helpers.h
 create mode 100644 block/export/vhost-user-blk-server.c
 create mode 100644 stubs/blk-exp-close-all.c
 create mode 100644 tests/qtest/libqos/vhost-user-blk.c
 create mode 100644 tests/qtest/vhost-user-blk-test.c
 create mode 100644 util/block-helpers.c
 create mode 100644 util/vhost-user-server.c

-- 
2.26.2

Comments

no-reply@patchew.org Oct. 12, 2020, 7 p.m. UTC | #1
Patchew URL: https://patchew.org/QEMU/20201012182800.157697-1-stefanha@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20201012182800.157697-1-stefanha@redhat.com
Subject: [PULL v2 00/30] Block patches

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20201012182800.157697-1-stefanha@redhat.com -> patchew/20201012182800.157697-1-stefanha@redhat.com
Switched to a new branch 'test'
0a960f8 iotests: add commit top->base cases to 274
2904098 block/io: fix bdrv_is_allocated_above
d4a7d3e block/io: bdrv_common_block_status_above: support bs == base
b49dc55 block/io: bdrv_common_block_status_above: support include_base
d855c38 block/io: fix bdrv_co_block_status_above
617ab32 tests/qtest: add multi-queue test case to vhost-user-blk-test
1ba3da8 block/export: add vhost-user-blk multi-queue support
c9cda63 block/export: add iothread and fixed-iothread options
8d7ed6c block: move block exports to libblockdev
c68e244 qemu-storage-daemon: avoid compiling blockdev_ss twice
05df47a util/vhost-user-server: use static library in meson.build
55e251f util/vhost-user-server: move header to include/
b1d101e block/export: convert vhost-user-blk server to block export API
21c3c13 block/export: report flush errors
559e3b3 util/vhost-user-server: rework vu_client_trip() coroutine lifecycle
c4ccc02 util/vhost-user-server: check EOF when reading payload
db49ac1 util/vhost-user-server: fix memory leak in vu_message_read()
5b01ca0 util/vhost-user-server: drop unused DevicePanicNotifier
1aab372 block/export: consolidate request structs into VuBlockReq
1df6ecd util/vhost-user-server: drop unnecessary watch deletion
a8933a8 util/vhost-user-server: drop unnecessary QOM cast
bd4edb0 util/vhost-user-server: s/fileds/fields/ typo fix
bec8cbf MAINTAINERS: Add vhost-user block device backend server maintainer
de2268c test: new qTest case to test the vhost-user-blk-server
ec6d467 block/export: vhost-user block device backend server
6ac349d block: move logical block size check function to a common utility function
9036dfc util/vhost-user-server: generic vhost user server
a70c2f0 libvhost-user: remove watch for kick_fd when de-initialize vu-dev
cea6c96 libvhost-user: Allow vu_message_read to be replaced
5023e8a block/nvme: Add driver statistics for access alignment and hw errors

=== OUTPUT BEGIN ===
1/30 Checking commit 5023e8a8bf04 (block/nvme: Add driver statistics for access alignment and hw errors)
2/30 Checking commit cea6c96deb4e (libvhost-user: Allow vu_message_read to be replaced)
WARNING: Block comments use a leading /* on a separate line
#130: FILE: contrib/libvhost-user/libvhost-user.h:395:
+    /* @read_msg: custom method to read vhost-user message

total: 0 errors, 1 warnings, 139 lines checked

Patch 2/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/30 Checking commit a70c2f01bb3f (libvhost-user: remove watch for kick_fd when de-initialize vu-dev)
4/30 Checking commit 9036dfc2c475 (util/vhost-user-server: generic vhost user server)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#33: 
new file mode 100644

WARNING: line over 80 characters
#85: FILE: util/vhost-user-server.c:48:
+    /* When this is set vu_client_trip will stop new processing vhost-user message */

total: 0 errors, 2 warnings, 500 lines checked

Patch 4/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/30 Checking commit 6ac349dc1fbf (block: move logical block size check function to a common utility function)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#85: 
new file mode 100644

total: 0 errors, 1 warnings, 129 lines checked

Patch 5/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/30 Checking commit ec6d467efa05 (block/export: vhost-user block device backend server)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#28: 
new file mode 100644

WARNING: line over 80 characters
#474: FILE: block/export/vhost-user-blk-server.c:442:
+        blk_remove_aio_context_notifier(vu_block_device->backend, blk_aio_attached,

ERROR: g_free(NULL) is safe this check is probably not required
#534: FILE: block/export/vhost-user-blk-server.c:502:
+    if (vus->node_name) {
+        g_free(vus->node_name);

total: 1 errors, 2 warnings, 714 lines checked

Patch 6/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

7/30 Checking commit de2268c7a097 (test: new qTest case to test the vhost-user-blk-server)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#69: 
new file mode 100644

WARNING: line over 80 characters
#188: FILE: tests/qtest/libqos/vhost-user-blk.c:115:
+    qos_node_create_driver("vhost-user-blk-device", vhost_user_blk_device_create);

total: 0 errors, 2 warnings, 1033 lines checked

Patch 7/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
8/30 Checking commit bec8cbf2870f (MAINTAINERS: Add vhost-user block device backend server maintainer)
9/30 Checking commit bd4edb0bfdb1 (util/vhost-user-server: s/fileds/fields/ typo fix)
10/30 Checking commit a8933a8cf288 (util/vhost-user-server: drop unnecessary QOM cast)
11/30 Checking commit 1df6ecdaba00 (util/vhost-user-server: drop unnecessary watch deletion)
12/30 Checking commit 1aab372a3e65 (block/export: consolidate request structs into VuBlockReq)
13/30 Checking commit 5b01ca030c4f (util/vhost-user-server: drop unused DevicePanicNotifier)
14/30 Checking commit db49ac1ba7b8 (util/vhost-user-server: fix memory leak in vu_message_read())
15/30 Checking commit c4ccc027a781 (util/vhost-user-server: check EOF when reading payload)
16/30 Checking commit 559e3b3faf66 (util/vhost-user-server: rework vu_client_trip() coroutine lifecycle)
17/30 Checking commit 21c3c139b052 (block/export: report flush errors)
18/30 Checking commit b1d101e37e29 (block/export: convert vhost-user-blk server to block export API)
WARNING: line over 80 characters
#858: FILE: util/vhost-user-server.c:415:
+        error_setg(errp, "Only socket address types 'unix' and 'fd' are supported");

total: 0 errors, 1 warnings, 758 lines checked

Patch 18/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
19/30 Checking commit 55e251f48bd8 (util/vhost-user-server: move header to include/)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#47: 
rename from util/vhost-user-server.h

total: 0 errors, 1 warnings, 29 lines checked

Patch 19/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
20/30 Checking commit 05df47ad1ea8 (util/vhost-user-server: use static library in meson.build)
21/30 Checking commit c68e244cd7f9 (qemu-storage-daemon: avoid compiling blockdev_ss twice)
22/30 Checking commit 8d7ed6caa410 (block: move block exports to libblockdev)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#131: 
new file mode 100644

total: 0 errors, 1 warnings, 90 lines checked

Patch 22/30 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
23/30 Checking commit c9cda634863c (block/export: add iothread and fixed-iothread options)
24/30 Checking commit 1ba3da805417 (block/export: add vhost-user-blk multi-queue support)
25/30 Checking commit 617ab3231ec4 (tests/qtest: add multi-queue test case to vhost-user-blk-test)
26/30 Checking commit d855c38a3530 (block/io: fix bdrv_co_block_status_above)
27/30 Checking commit b49dc559331f (block/io: bdrv_common_block_status_above: support include_base)
28/30 Checking commit d4a7d3e520b9 (block/io: bdrv_common_block_status_above: support bs == base)
29/30 Checking commit 2904098296a3 (block/io: fix bdrv_is_allocated_above)
30/30 Checking commit 0a960f82952e (iotests: add commit top->base cases to 274)
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20201012182800.157697-1-stefanha@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Peter Maydell Oct. 12, 2020, 9:48 p.m. UTC | #2
On Mon, 12 Oct 2020 at 19:28, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>

> The following changes since commit 2387df497b4b4bcf754eb7398edca82889e2ef54:

>

>   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' into staging (2020-10-12 11:29:42 +0100)

>

> are available in the Git repository at:

>

>   https://gitlab.com/stefanha/qemu.git tags/block-pull-request

>

> for you to fetch changes up to 3664ec6bbe236126b79d251d4037889e7181ab55:

>

>   iotests: add commit top->base cases to 274 (2020-10-12 16:47:58 +0100)

>

> ----------------------------------------------------------------

> Pull request

>

> v2:

>  * Rebase and resolve conflict with commit 029a88c9a7e3 ("qemu-nbd: Honor

>    SIGINT and SIGHUP") [Peter]

>


Build failures, OSX and the BSDs:

Compiling C object
contrib/libvhost-user/libvhost-user.a.p/libvhost-user-glib.c.o
../../contrib/libvhost-user/libvhost-user.c:27:10: fatal error:
'sys/eventfd.h' file not found
#include <sys/eventfd.h>
         ^~~~~~~~~~~~~~~
In file included from ../../contrib/libvhost-user/libvhost-user-glib.c:17:
In file included from ../../contrib/libvhost-user/libvhost-user-glib.h:19:
../../contrib/libvhost-user/libvhost-user.h:21:10: fatal error:
'linux/vhost.h' file not found
#include <linux/vhost.h>
         ^~~~~~~~~~~~~~~
1 error generated.


Build failure, 32-bit:

../../util/vhost-user-server.c: In function 'vu_message_read':
../../util/vhost-user-server.c:119:30: error: format '%lu' expects
argument of type 'long unsigned int', but argument 3 has type 'size_t
{aka unsigned int}' [-Werror=format=]
                 error_report("A maximum of %zu fds are allowed, "
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../util/vhost-user-server.c:121:39:
                              max_fds, vmsg->fd_num + nfds);
                                       ~~~~~~~~~~~~~~~~~~~
../../util/vhost-user-server.c:120:45: note: format string is defined here
                              "however got %lu fds now",
                                           ~~^
                                           %u

(you might also want to wordsmith the English in that error message)

Build failure, Windows:

../../qemu-nbd.c:158:5: error: "CONFIG_POSIX" is not defined [-Werror=undef]
 #if CONFIG_POSIX
     ^

'make check' failure, s390x and ppc64 (ie big-endian host):

ERROR:../../tests/qtest/vhost-user-blk-test.c:448:idx: assertion
failed (capacity == TEST_IMAGE_SIZE / 512): (2199023255552 == 131072)
ERROR qtest-i386: qos-test - Bail out!
ERROR:../../tests/qtest/vhost-user-blk-test.c:448:idx: assertion
failed (capacity == TEST_IMAGE_SIZE / 512): (2199023255552 == 131072)

I also got this on aarch64 host:

ERROR:../../tests/qtest/boot-sector.c:161:boot_sector_test: assertion
failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)
ERROR qtest-s390x: cdrom-test - Bail out!
ERROR:../../tests/qtest/boot-sector.c:161:boot_sector_test: assertion
failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)

but it might be an unrelated intermittent.

Also a hang on x86-64 host running qtest-ppc64 qos-test, ditto.

thanks
-- PMM
Eric Blake Oct. 13, 2020, 12:05 a.m. UTC | #3
On 10/12/20 4:48 PM, Peter Maydell wrote:

> 

> Build failures, OSX and the BSDs:

> 


I'll let you find and fix those...

> Build failure, Windows:

> 

> ../../qemu-nbd.c:158:5: error: "CONFIG_POSIX" is not defined [-Werror=undef]

>   #if CONFIG_POSIX

>       ^

> 


but this one is easy.  In 22/30 block: move block exports to 
libblockdev, replace '#if CONFIG_POSIX' with '#ifdef CONFIG_POSIX' 
around qemu_system_killed().

> 'make check' failure, s390x and ppc64 (ie big-endian host):

> 

> ERROR:../../tests/qtest/vhost-user-blk-test.c:448:idx: assertion

> failed (capacity == TEST_IMAGE_SIZE / 512): (2199023255552 == 131072)

> ERROR qtest-i386: qos-test - Bail out!

> ERROR:../../tests/qtest/vhost-user-blk-test.c:448:idx: assertion

> failed (capacity == TEST_IMAGE_SIZE / 512): (2199023255552 == 131072)

> 

> I also got this on aarch64 host:

> 

> ERROR:../../tests/qtest/boot-sector.c:161:boot_sector_test: assertion

> failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)

> ERROR qtest-s390x: cdrom-test - Bail out!

> ERROR:../../tests/qtest/boot-sector.c:161:boot_sector_test: assertion

> failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)

> 

> but it might be an unrelated intermittent.

> 

> Also a hang on x86-64 host running qtest-ppc64 qos-test, ditto.

> 

> thanks

> -- PMM

> 


-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org
Stefan Hajnoczi Oct. 20, 2020, 3:19 p.m. UTC | #4
On Mon, Oct 12, 2020 at 10:48:17PM +0100, Peter Maydell wrote:
> On Mon, 12 Oct 2020 at 19:28, Stefan Hajnoczi <stefanha@redhat.com> wrote:

> >

> > The following changes since commit 2387df497b4b4bcf754eb7398edca82889e2ef54:

> >

> >   Merge remote-tracking branch 'remotes/armbru/tags/pull-qapi-2020-10-10' into staging (2020-10-12 11:29:42 +0100)

> >

> > are available in the Git repository at:

> >

> >   https://gitlab.com/stefanha/qemu.git tags/block-pull-request

> >

> > for you to fetch changes up to 3664ec6bbe236126b79d251d4037889e7181ab55:

> >

> >   iotests: add commit top->base cases to 274 (2020-10-12 16:47:58 +0100)

> >

> > ----------------------------------------------------------------

> > Pull request

> >

> > v2:

> >  * Rebase and resolve conflict with commit 029a88c9a7e3 ("qemu-nbd: Honor

> >    SIGINT and SIGHUP") [Peter]

> >

> 

> Build failures, OSX and the BSDs:

> 

> Compiling C object

> contrib/libvhost-user/libvhost-user.a.p/libvhost-user-glib.c.o

> ../../contrib/libvhost-user/libvhost-user.c:27:10: fatal error:

> 'sys/eventfd.h' file not found

> #include <sys/eventfd.h>

>          ^~~~~~~~~~~~~~~

> In file included from ../../contrib/libvhost-user/libvhost-user-glib.c:17:

> In file included from ../../contrib/libvhost-user/libvhost-user-glib.h:19:

> ../../contrib/libvhost-user/libvhost-user.h:21:10: fatal error:

> 'linux/vhost.h' file not found

> #include <linux/vhost.h>

>          ^~~~~~~~~~~~~~~

> 1 error generated.

> 

> 

> Build failure, 32-bit:

> 

> ../../util/vhost-user-server.c: In function 'vu_message_read':

> ../../util/vhost-user-server.c:119:30: error: format '%lu' expects

> argument of type 'long unsigned int', but argument 3 has type 'size_t

> {aka unsigned int}' [-Werror=format=]

>                  error_report("A maximum of %zu fds are allowed, "

>                               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

> ../../util/vhost-user-server.c:121:39:

>                               max_fds, vmsg->fd_num + nfds);

>                                        ~~~~~~~~~~~~~~~~~~~

> ../../util/vhost-user-server.c:120:45: note: format string is defined here

>                               "however got %lu fds now",

>                                            ~~^

>                                            %u

> 

> (you might also want to wordsmith the English in that error message)

> 

> Build failure, Windows:

> 

> ../../qemu-nbd.c:158:5: error: "CONFIG_POSIX" is not defined [-Werror=undef]

>  #if CONFIG_POSIX

>      ^

> 

> 'make check' failure, s390x and ppc64 (ie big-endian host):

> 

> ERROR:../../tests/qtest/vhost-user-blk-test.c:448:idx: assertion

> failed (capacity == TEST_IMAGE_SIZE / 512): (2199023255552 == 131072)

> ERROR qtest-i386: qos-test - Bail out!

> ERROR:../../tests/qtest/vhost-user-blk-test.c:448:idx: assertion

> failed (capacity == TEST_IMAGE_SIZE / 512): (2199023255552 == 131072)

> 

> I also got this on aarch64 host:

> 

> ERROR:../../tests/qtest/boot-sector.c:161:boot_sector_test: assertion

> failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)

> ERROR qtest-s390x: cdrom-test - Bail out!

> ERROR:../../tests/qtest/boot-sector.c:161:boot_sector_test: assertion

> failed (signature == SIGNATURE): (0x00000000 == 0x0000dead)

> 

> but it might be an unrelated intermittent.

> 

> Also a hang on x86-64 host running qtest-ppc64 qos-test, ditto.


Thanks for the info. I will send a v2.

Stefan