| Message ID | 20201013190506.3325-1-jandryuk@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | hw/xen: Set suppress-vmdesc for Xen machines | expand |
On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote: > xen-save-devices-state doesn't currently generate a vmdesc, so restore > always triggers "Expected vmdescription section, but got 0". This is > not a problem when restore comes from a file. However, when QEMU runs > in a linux stubdom and comes over a console, EOF is not received. This > causes a delay restoring - though it does restore. > > Setting suppress-vmdesc skips looking for the vmdesc during restore and > avoids the wait. suppress-vmdesc is only used during restore, right? So starting a guest without it, saving the guest and restoring the guest with suppress-vmdesc=on added will work as intended? (I'm checking that migration across update of QEMU will work.) Thanks, -- Anthony PERARD
On Fri, Oct 16, 2020 at 11:38 AM Anthony PERARD <anthony.perard@citrix.com> wrote: > > On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote: > > xen-save-devices-state doesn't currently generate a vmdesc, so restore > > always triggers "Expected vmdescription section, but got 0". This is > > not a problem when restore comes from a file. However, when QEMU runs > > in a linux stubdom and comes over a console, EOF is not received. This > > causes a delay restoring - though it does restore. > > > > Setting suppress-vmdesc skips looking for the vmdesc during restore and > > avoids the wait. > > suppress-vmdesc is only used during restore, right? So starting a guest > without it, saving the guest and restoring the guest with > suppress-vmdesc=on added will work as intended? (I'm checking that migration > across update of QEMU will work.) vmdesc is a json description of the migration stream that comes after the QEMU migration stream. For our purposes, <migration stream><vmdesc json blob>. Normal QEMU savevm will generate it, unless suppress-vmdesc is set. QEMU restore will read it because: "Try to read in the VMDESC section as well, so that dumping tools that intercept our migration stream have the chance to see it." Xen save does not go through savevm, but instead xen-save-devices-state, which is a subset of the QEMU savevm. It skips RAM since that is read out through Xen interfaces. Xen uses xen-load-devices-state to restore device state. That goes through the common qemu_loadvm_state which tries to read the vmdesc stream. For Xen, yes, suppress-vmdesc only matters for the restore case, and it suppresses the attempt to read the vmdesc. I think every Xen restore currently has "Expected vmdescription section, but got -1" in the -dm.log since the vmdesc is missing. I have not tested restoring across this change, but since it just controls reading and discarding the vmdesc stream, I don't think it will break migration across update. Regards, Jason
On Fri, Oct 16, 2020 at 12:01:47PM -0400, Jason Andryuk wrote: > On Fri, Oct 16, 2020 at 11:38 AM Anthony PERARD > <anthony.perard@citrix.com> wrote: > > > > On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote: > > > xen-save-devices-state doesn't currently generate a vmdesc, so restore > > > always triggers "Expected vmdescription section, but got 0". This is > > > not a problem when restore comes from a file. However, when QEMU runs > > > in a linux stubdom and comes over a console, EOF is not received. This > > > causes a delay restoring - though it does restore. > > > > > > Setting suppress-vmdesc skips looking for the vmdesc during restore and > > > avoids the wait. > > > > suppress-vmdesc is only used during restore, right? So starting a guest > > without it, saving the guest and restoring the guest with > > suppress-vmdesc=on added will work as intended? (I'm checking that migration > > across update of QEMU will work.) > > vmdesc is a json description of the migration stream that comes after > the QEMU migration stream. For our purposes, <migration > stream><vmdesc json blob>. Normal QEMU savevm will generate it, > unless suppress-vmdesc is set. QEMU restore will read it because: > "Try to read in the VMDESC section as well, so that dumping tools that > intercept our migration stream have the chance to see it." > > Xen save does not go through savevm, but instead > xen-save-devices-state, which is a subset of the QEMU savevm. It > skips RAM since that is read out through Xen interfaces. Xen uses > xen-load-devices-state to restore device state. That goes through the > common qemu_loadvm_state which tries to read the vmdesc stream. > > For Xen, yes, suppress-vmdesc only matters for the restore case, and > it suppresses the attempt to read the vmdesc. I think every Xen > restore currently has "Expected vmdescription section, but got -1" in > the -dm.log since the vmdesc is missing. I have not tested restoring > across this change, but since it just controls reading and discarding > the vmdesc stream, I don't think it will break migration across > update. Thanks for the explanation. Acked-by: Anthony PERARD <anthony.perard@citrix.com> Do you think you could send a patch for libxl as well? Since libxl in some cases may use the "pc machine instead of "xenfv". I can send the patch otherwise. Cheers,
On Fri, Oct 16, 2020 at 12:44 PM Anthony PERARD <anthony.perard@citrix.com> wrote: > > On Fri, Oct 16, 2020 at 12:01:47PM -0400, Jason Andryuk wrote: > > On Fri, Oct 16, 2020 at 11:38 AM Anthony PERARD > > <anthony.perard@citrix.com> wrote: > > > > > > On Tue, Oct 13, 2020 at 03:05:06PM -0400, Jason Andryuk wrote: > > > > xen-save-devices-state doesn't currently generate a vmdesc, so restore > > > > always triggers "Expected vmdescription section, but got 0". This is > > > > not a problem when restore comes from a file. However, when QEMU runs > > > > in a linux stubdom and comes over a console, EOF is not received. This > > > > causes a delay restoring - though it does restore. > > > > > > > > Setting suppress-vmdesc skips looking for the vmdesc during restore and > > > > avoids the wait. > > > > > > suppress-vmdesc is only used during restore, right? So starting a guest > > > without it, saving the guest and restoring the guest with > > > suppress-vmdesc=on added will work as intended? (I'm checking that migration > > > across update of QEMU will work.) > > > > vmdesc is a json description of the migration stream that comes after > > the QEMU migration stream. For our purposes, <migration > > stream><vmdesc json blob>. Normal QEMU savevm will generate it, > > unless suppress-vmdesc is set. QEMU restore will read it because: > > "Try to read in the VMDESC section as well, so that dumping tools that > > intercept our migration stream have the chance to see it." > > > > Xen save does not go through savevm, but instead > > xen-save-devices-state, which is a subset of the QEMU savevm. It > > skips RAM since that is read out through Xen interfaces. Xen uses > > xen-load-devices-state to restore device state. That goes through the > > common qemu_loadvm_state which tries to read the vmdesc stream. > > > > For Xen, yes, suppress-vmdesc only matters for the restore case, and > > it suppresses the attempt to read the vmdesc. I think every Xen > > restore currently has "Expected vmdescription section, but got -1" in > > the -dm.log since the vmdesc is missing. I have not tested restoring > > across this change, but since it just controls reading and discarding > > the vmdesc stream, I don't think it will break migration across > > update. > > Thanks for the explanation. > > Acked-by: Anthony PERARD <anthony.perard@citrix.com> > > Do you think you could send a patch for libxl as well? Since libxl in > some cases may use the "pc machine instead of "xenfv". I can send the > patch otherwise. I should be able to, yes. Regards, Jason
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c index 3c2ae0612b..0cf22a57ad 100644 --- a/hw/i386/pc_piix.c +++ b/hw/i386/pc_piix.c @@ -987,7 +987,7 @@ static void xenfv_4_2_machine_options(MachineClass *m) pc_i440fx_4_2_machine_options(m); m->desc = "Xen Fully-virtualized PC"; m->max_cpus = HVM_MAX_VCPUS; - m->default_machine_opts = "accel=xen"; + m->default_machine_opts = "accel=xen,suppress-vmdesc=on"; } DEFINE_PC_MACHINE(xenfv_4_2, "xenfv-4.2", pc_xen_hvm_init, @@ -999,7 +999,7 @@ static void xenfv_3_1_machine_options(MachineClass *m) m->desc = "Xen Fully-virtualized PC"; m->alias = "xenfv"; m->max_cpus = HVM_MAX_VCPUS; - m->default_machine_opts = "accel=xen"; + m->default_machine_opts = "accel=xen,suppress-vmdesc=on"; } DEFINE_PC_MACHINE(xenfv, "xenfv-3.1", pc_xen_hvm_init,
xen-save-devices-state doesn't currently generate a vmdesc, so restore always triggers "Expected vmdescription section, but got 0". This is not a problem when restore comes from a file. However, when QEMU runs in a linux stubdom and comes over a console, EOF is not received. This causes a delay restoring - though it does restore. Setting suppress-vmdesc skips looking for the vmdesc during restore and avoids the wait. The other approach would be generate a vmdesc in qemu_save_device_state. Since COLO shared that function, and the vmdesc is just discarded on restore, we choose to skip it. Reported-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com> Signed-off-by: Jason Andryuk <jandryuk@gmail.com> --- hw/i386/pc_piix.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)