mbox series

[v4,0/5] block: add block-dirty-bitmap-populate job

Message ID 20200902181831.2570048-1-eblake@redhat.com
Headers show
Series block: add block-dirty-bitmap-populate job | expand

Message

Eric Blake Sept. 2, 2020, 6:18 p.m. UTC
This is NOT the final version of this patch series, but I'm posting it
to revive conversation on the topic while fixing it to compile on top
of meson changes.

v3 was: https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06258.html

001/5:[0025] [FC] 'block: add bitmap-populate job'
002/5:[----] [--] 'blockdev: combine DriveBackupState and BlockdevBackupState'
003/5:[0004] [FC] 'qmp: expose block-dirty-bitmap-populate'
004/5:[----] [--] 'iotests: move bitmap helpers into their own file'
005/5:[----] [-C] 'iotests: add 298 for block-dirty-bitmap-populate'

I'm still trying to find the right QAPI contract (affects patch 1 and
3 for the qapi, and 5 for invoking the command in iotests), but right
now, I'm leaning towards:

{ "execute": "block-dirty-bitmap-populate", "arguments": {
    "job-id": "job0", "node": "target_node", "name": "target_bitmap",
    "sources": [ { "pattern": "allocation", "node": "from_node" } ] } }

which allows expansion into multiple sources, so that we can combine a
populate action with a bitmap merge rather than having to do those as
separate commands, like:

{ "execute": "block-dirty-bitmap-populate", "arguments": {
    "job-id": "job0", "node": "target_node", "name": "target_bitmap",
    "sources": [ { "pattern": "allocation", "node": "from_node" },
                 { "pattern": "bitmap", "node": "from_node",
		     "bitmap": "from_bitmap" } ] } }

John Snow (5):
  block: add bitmap-populate job
  blockdev: combine DriveBackupState and BlockdevBackupState
  qmp: expose block-dirty-bitmap-populate
  iotests: move bitmap helpers into their own file
  iotests: add 298 for block-dirty-bitmap-populate

 qapi/block-core.json          |   66 +
 qapi/job.json                 |    6 +-
 qapi/transaction.json         |    2 +
 include/block/block.h         |    1 +
 include/block/block_int.h     |   21 +
 block/bitmap-populate.c       |  207 ++
 blockdev.c                    |  104 +-
 blockjob.c                    |    3 +-
 MAINTAINERS                   |    1 +
 block/meson.build             |    1 +
 tests/qemu-iotests/257        |  110 +-
 tests/qemu-iotests/298        |  232 ++
 tests/qemu-iotests/298.out    | 4544 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/bitmaps.py |  131 +
 tests/qemu-iotests/group      |    1 +
 15 files changed, 5301 insertions(+), 129 deletions(-)
 create mode 100644 block/bitmap-populate.c
 create mode 100755 tests/qemu-iotests/298
 create mode 100644 tests/qemu-iotests/298.out
 create mode 100644 tests/qemu-iotests/bitmaps.py

Comments

Vladimir Sementsov-Ogievskiy Sept. 3, 2020, 6:50 a.m. UTC | #1
02.09.2020 21:18, Eric Blake wrote:
> This is NOT the final version of this patch series, but I'm posting it
> to revive conversation on the topic while fixing it to compile on top
> of meson changes.
> 
> v3 was: https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06258.html
> 
> 001/5:[0025] [FC] 'block: add bitmap-populate job'
> 002/5:[----] [--] 'blockdev: combine DriveBackupState and BlockdevBackupState'
> 003/5:[0004] [FC] 'qmp: expose block-dirty-bitmap-populate'
> 004/5:[----] [--] 'iotests: move bitmap helpers into their own file'
> 005/5:[----] [-C] 'iotests: add 298 for block-dirty-bitmap-populate'
> 
> I'm still trying to find the right QAPI contract (affects patch 1 and
> 3 for the qapi, and 5 for invoking the command in iotests), but right
> now, I'm leaning towards:
> 
> { "execute": "block-dirty-bitmap-populate", "arguments": {
>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",
>      "sources": [ { "pattern": "allocation", "node": "from_node" } ] } }
> 
> which allows expansion into multiple sources, so that we can combine a
> populate action with a bitmap merge rather than having to do those as
> separate commands, like:
> 
> { "execute": "block-dirty-bitmap-populate", "arguments": {
>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",
>      "sources": [ { "pattern": "allocation", "node": "from_node" },
>                   { "pattern": "bitmap", "node": "from_node",
> 		     "bitmap": "from_bitmap" } ] } }

(I see, this is not realized in v4)

What about multiple targets as well, as we discussed in v3?

> 
> John Snow (5):
>    block: add bitmap-populate job
>    blockdev: combine DriveBackupState and BlockdevBackupState
>    qmp: expose block-dirty-bitmap-populate
>    iotests: move bitmap helpers into their own file
>    iotests: add 298 for block-dirty-bitmap-populate
> 
>   qapi/block-core.json          |   66 +
>   qapi/job.json                 |    6 +-
>   qapi/transaction.json         |    2 +
>   include/block/block.h         |    1 +
>   include/block/block_int.h     |   21 +
>   block/bitmap-populate.c       |  207 ++
>   blockdev.c                    |  104 +-
>   blockjob.c                    |    3 +-
>   MAINTAINERS                   |    1 +
>   block/meson.build             |    1 +
>   tests/qemu-iotests/257        |  110 +-
>   tests/qemu-iotests/298        |  232 ++
>   tests/qemu-iotests/298.out    | 4544 +++++++++++++++++++++++++++++++++
>   tests/qemu-iotests/bitmaps.py |  131 +
>   tests/qemu-iotests/group      |    1 +
>   15 files changed, 5301 insertions(+), 129 deletions(-)
>   create mode 100644 block/bitmap-populate.c
>   create mode 100755 tests/qemu-iotests/298
>   create mode 100644 tests/qemu-iotests/298.out
>   create mode 100644 tests/qemu-iotests/bitmaps.py
>
Vladimir Sementsov-Ogievskiy Sept. 3, 2020, 6:56 a.m. UTC | #2
02.09.2020 21:18, Eric Blake wrote:
> From: John Snow<jsnow@redhat.com>
> 
> Signed-off-by: John Snow<jsnow@redhat.com>
> Message-Id:<20200514034922.24834-5-jsnow@redhat.com>
> Signed-off-by: Eric Blake<eblake@redhat.com>

You forget my:

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Vladimir Sementsov-Ogievskiy March 19, 2021, 8:04 p.m. UTC | #3
Hi!

Any plans on v5? Now I have a request for this feature.. If no plans, I can take care of it and make a v5

02.09.2020 21:18, Eric Blake wrote:
> This is NOT the final version of this patch series, but I'm posting it

> to revive conversation on the topic while fixing it to compile on top

> of meson changes.

> 

> v3 was:https://lists.gnu.org/archive/html/qemu-devel/2020-06/msg06258.html

> 

> 001/5:[0025] [FC] 'block: add bitmap-populate job'

> 002/5:[----] [--] 'blockdev: combine DriveBackupState and BlockdevBackupState'

> 003/5:[0004] [FC] 'qmp: expose block-dirty-bitmap-populate'

> 004/5:[----] [--] 'iotests: move bitmap helpers into their own file'

> 005/5:[----] [-C] 'iotests: add 298 for block-dirty-bitmap-populate'

> 

> I'm still trying to find the right QAPI contract (affects patch 1 and

> 3 for the qapi, and 5 for invoking the command in iotests), but right

> now, I'm leaning towards:

> 

> { "execute": "block-dirty-bitmap-populate", "arguments": {

>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",

>      "sources": [ { "pattern": "allocation", "node": "from_node" } ] } }

> 

> which allows expansion into multiple sources, so that we can combine a

> populate action with a bitmap merge rather than having to do those as

> separate commands, like:

> 

> { "execute": "block-dirty-bitmap-populate", "arguments": {

>      "job-id": "job0", "node": "target_node", "name": "target_bitmap",

>      "sources": [ { "pattern": "allocation", "node": "from_node" },

>                   { "pattern": "bitmap", "node": "from_node",

> 		     "bitmap": "from_bitmap" } ] } }



-- 
Best regards,
Vladimir