| Message ID | 1398681696-2773-12-git-send-email-ian.campbell@citrix.com |
|---|---|
| State | New |
| Headers | show |
Ian Campbell writes ("[PATCH OSSTEST v2 12/15] distros: support PV guest install from Debian netinst media."):
> The netinst media are iso images containing a base Debian install and some
> (image size dependent) additional tasks.
>
> On x86 the "multiarch" iso flavour contains a Xen capable kernel for both i386
> and amd64 so use that.
>
> This adds support for two classes of ISO, the CD sized ones which are built
> nightly and the DVD sized ones which are built weekly.
>
> The images are downloaded using jigdo which sources the majority of the data
> from a local Debian mirror, for this reason I have not worried about the fact
> that the i386 and amd64 tests are downloading the same thing (adding a specific
> download job would require finding up to 4GB of scratch space for each flight).
>
> The ISOs booted using pygrub which can extract the kernel and initrd from a ISO
> image. The resulting guests are also booted with pygrub since the pv-grub-menu
> package is not available on the ISO images and we have pvgrub coverage from the
> netboot tests.
>
> Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
> ---
> Osstest/Debian.pm | 7 +++++--
> make-distros-flight | 22 ++++++++++++++++++++
> ts-debian-di-install | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 84 insertions(+), 2 deletions(-)
>
> diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
> index ad478ab..b149e8d 100644
> --- a/Osstest/Debian.pm
> +++ b/Osstest/Debian.pm
> @@ -433,8 +433,6 @@ sub preseed_base ($$;@) {
> $extra_packages ||= '';
>
> my $preseed= (<<END);
> -d-i mirror/suite string $suite
> -
> d-i debian-installer/locale string en_GB
> d-i console-keymaps-at/keymap select gb
> d-i keyboard-configuration/xkb-keymap string en_GB
> @@ -502,6 +500,11 @@ $xopts{ExtraPreseed}
>
> END
>
> + # For CDROM the suite is part of the image
> + $preseed .= <<END unless $xopts{CDROM};
> +d-i mirror/suite string $suite
> +END
> +
> # deb http://ftp.debian.org/debian/ wheezy-backports main
> $preseed .= <<END if $xopts{EnableBackports};
> d-i apt-setup/local0/repository string http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main
> diff --git a/make-distros-flight b/make-distros-flight
> index 244d582..a679c61 100755
> --- a/make-distros-flight
> +++ b/make-distros-flight
> @@ -86,6 +86,19 @@ test_do_one_netboot () {
> all_hostflags=$most_hostflags
> }
>
> +test_do_one_netinst () {
> + # Always pygrub since no pv-grub-menu on CD
> + job_create_test test-$xenarch$kern-$dom0arch-$domU-$cd-netinst \
> + test-debian-di xl $xenarch $dom0arch \
> + kernbuildjob=${bfi}build-$dom0arch-$kernbuild \
> + debian_arch=$domU \
> + debian_cd=$cd \
> + debian_method=netinst \
> + debian_bootloader=pygrub \
> + all_hostflags=$most_hostflags
> +
> +}
> +
> test_matrix_do_one () {
> case ${xenarch} in
> amd64) domUarches="amd64 i386";;
> @@ -104,6 +117,15 @@ test_matrix_do_one () {
>
> done
>
> + for cd in current weekly ; do
> + case ${domU}_${dist} in
> + armhf_*) continue;; # No iso targets for armhf
> + *) ;;
> + esac
> +
> + test_do_one_netinst
> +
> + done
> done
> }
>
> diff --git a/ts-debian-di-install b/ts-debian-di-install
> index 5ed15e4..56feee6 100755
> --- a/ts-debian-di-install
> +++ b/ts-debian-di-install
> @@ -46,6 +46,53 @@ sub prep () {
> target_cmd_root($ho, "umount $gho->{Lvdev} ||:");
> }
>
> +sub setup_netinst($$$)
> +{
...
> + # Use the MD5SUMs file as an index
> + logm("Fetch index from $baseurl/MD5SUMS");
> + open C, "curl $baseurl/MD5SUMS|" or die "curl: $!";
This needs to be done in a way that has a timeout. I don't know what
curl's default is but I think it would be best to arrange to use cmd
in TestSupport.pm somehow. (Maybe directly, in which case it needs to
be exported.)
> + target_cmd_root($ho, <<END, 3600);
> + cd $didir && jigdo-lite --noask $netinst_jigdo
> +END
Can you indent the script line so this is easier to read ?
> + store_runvar("$gho->{Guest}_netinst_jigdo", $netinst_jigdo);
> +
> + return (<<END, "\"file:$didir/$filebase.iso,xvdd:cdrom,r\",");
> +bootloader = "pygrub"
> +bootloader_args = ["--kernel=$props->{IsoPath}/vmlinuz", "--ramdisk=$props->{IsoPath}/initrd.gz"]
Could be wrapped.
Thanks,
Ian.
On Fri, 2014-05-02 at 15:01 +0100, Ian Jackson wrote: > Ian Campbell writes ("[PATCH OSSTEST v2 12/15] distros: support PV guest install from Debian netinst media."): > > +sub setup_netinst($$$) > > +{ > ... > > + # Use the MD5SUMs file as an index > > + logm("Fetch index from $baseurl/MD5SUMS"); > > + open C, "curl $baseurl/MD5SUMS|" or die "curl: $!"; > > This needs to be done in a way that has a timeout. I don't know what > curl's default is but I think it would be best to arrange to use cmd > in TestSupport.pm somehow. (Maybe directly, in which case it needs to > be exported.) I'll investigate. Is doing this from the test controller OK, moving it to the test dom0 would nicely fix thistoo. > > + target_cmd_root($ho, <<END, 3600); > > + cd $didir && jigdo-lite --noask $netinst_jigdo > > +END > > Can you indent the script line so this is easier to read ? Yes. > > + store_runvar("$gho->{Guest}_netinst_jigdo", $netinst_jigdo); > > + > > + return (<<END, "\"file:$didir/$filebase.iso,xvdd:cdrom,r\","); > > +bootloader = "pygrub" > > +bootloader_args = ["--kernel=$props->{IsoPath}/vmlinuz", "--ramdisk=$props->{IsoPath}/initrd.gz"] > > Could be wrapped. Ack.
Ian Campbell writes ("Re: [PATCH OSSTEST v2 12/15] distros: support PV guest install from Debian netinst media."):
> On Fri, 2014-05-02 at 15:01 +0100, Ian Jackson wrote:
> > This needs to be done in a way that has a timeout. I don't know what
> > curl's default is but I think it would be best to arrange to use cmd
> > in TestSupport.pm somehow. (Maybe directly, in which case it needs to
> > be exported.)
>
> I'll investigate.
>
> Is doing this from the test controller OK, moving it to the test dom0
> would nicely fix thistoo.
I think it's fine to do it from either.
Thanks,
Ian.
diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm index ad478ab..b149e8d 100644 --- a/Osstest/Debian.pm +++ b/Osstest/Debian.pm @@ -433,8 +433,6 @@ sub preseed_base ($$;@) { $extra_packages ||= ''; my $preseed= (<<END); -d-i mirror/suite string $suite - d-i debian-installer/locale string en_GB d-i console-keymaps-at/keymap select gb d-i keyboard-configuration/xkb-keymap string en_GB @@ -502,6 +500,11 @@ $xopts{ExtraPreseed} END + # For CDROM the suite is part of the image + $preseed .= <<END unless $xopts{CDROM}; +d-i mirror/suite string $suite +END + # deb http://ftp.debian.org/debian/ wheezy-backports main $preseed .= <<END if $xopts{EnableBackports}; d-i apt-setup/local0/repository string http://$c{DebianMirrorHost}/$c{DebianMirrorSubpath} $suite-backports main diff --git a/make-distros-flight b/make-distros-flight index 244d582..a679c61 100755 --- a/make-distros-flight +++ b/make-distros-flight @@ -86,6 +86,19 @@ test_do_one_netboot () { all_hostflags=$most_hostflags } +test_do_one_netinst () { + # Always pygrub since no pv-grub-menu on CD + job_create_test test-$xenarch$kern-$dom0arch-$domU-$cd-netinst \ + test-debian-di xl $xenarch $dom0arch \ + kernbuildjob=${bfi}build-$dom0arch-$kernbuild \ + debian_arch=$domU \ + debian_cd=$cd \ + debian_method=netinst \ + debian_bootloader=pygrub \ + all_hostflags=$most_hostflags + +} + test_matrix_do_one () { case ${xenarch} in amd64) domUarches="amd64 i386";; @@ -104,6 +117,15 @@ test_matrix_do_one () { done + for cd in current weekly ; do + case ${domU}_${dist} in + armhf_*) continue;; # No iso targets for armhf + *) ;; + esac + + test_do_one_netinst + + done done } diff --git a/ts-debian-di-install b/ts-debian-di-install index 5ed15e4..56feee6 100755 --- a/ts-debian-di-install +++ b/ts-debian-di-install @@ -46,6 +46,53 @@ sub prep () { target_cmd_root($ho, "umount $gho->{Lvdev} ||:"); } +sub setup_netinst($$$) +{ + my ($didir, $arch, $cd) = @_; + my %arch_props = ( + amd64 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.amd/xen" }, + i386 => { PathArch => "multi-arch", FileArch => "amd64-i386", IsoPath => "/install.386/xen" }, + armhf => { PathArch => "armhf", FileArch => "armhf", IsoPath => "/install.armhf" } + ); + my $props = $arch_props{$arch} or die "Unknown arch $arch"; + + target_install_packages($ho, qw(jigdo-file)); + + my $baseurl = $cd eq "current" ? + "http://cdimage.debian.org/debian-cd/current/$props->{PathArch}/jigdo-cd" : + "http://cdimage.debian.org/cdimage/weekly-builds/$props->{PathArch}/jigdo-cd"; + + my $filebase; + + # Use the MD5SUMs file as an index + logm("Fetch index from $baseurl/MD5SUMS"); + open C, "curl $baseurl/MD5SUMS|" or die "curl: $!"; + while(<C>) { + m/^[0-9a-f]{32} (debian-.*-$props->{FileArch}-netinst)\.iso$/ or next; + $filebase = $1; + last; + } + close(C); + + die unless $filebase; + + logm("Downloading $baseurl/$filebase.jigdo"); + # Jigdo seems to use /etc/apt/sources.list or something, so this + # just works using the already configured mirror without + # additional configuration, which is good because there doesn't + # seem to be any support for such things, at least in Squeeze. + my $netinst_jigdo = "$baseurl/$filebase.jigdo"; + target_cmd_root($ho, <<END, 3600); + cd $didir && jigdo-lite --noask $netinst_jigdo +END + store_runvar("$gho->{Guest}_netinst_jigdo", $netinst_jigdo); + + return (<<END, "\"file:$didir/$filebase.iso,xvdd:cdrom,r\","); +bootloader = "pygrub" +bootloader_args = ["--kernel=$props->{IsoPath}/vmlinuz", "--ramdisk=$props->{IsoPath}/initrd.gz"] +END +} + sub setup_netboot($$$) { my ($didir, $arch, $suite) = @_; @@ -111,6 +158,16 @@ END $extra_disk = ""; } + elsif ($method eq "netinst" ) + { + my $cd = $r{"$gho->{Guest}_cd"}; + + logm("$method $cd/$arch"); + + ($method_cfg,$extra_disk) = setup_netinst("/root/di", $arch, $cd); + + $ps_url = preseed_create_guest($gho, '', CDROM=>1); + } else { die "$method";
The netinst media are iso images containing a base Debian install and some (image size dependent) additional tasks. On x86 the "multiarch" iso flavour contains a Xen capable kernel for both i386 and amd64 so use that. This adds support for two classes of ISO, the CD sized ones which are built nightly and the DVD sized ones which are built weekly. The images are downloaded using jigdo which sources the majority of the data from a local Debian mirror, for this reason I have not worried about the fact that the i386 and amd64 tests are downloading the same thing (adding a specific download job would require finding up to 4GB of scratch space for each flight). The ISOs booted using pygrub which can extract the kernel and initrd from a ISO image. The resulting guests are also booted with pygrub since the pv-grub-menu package is not available on the ISO images and we have pvgrub coverage from the netboot tests. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> --- Osstest/Debian.pm | 7 +++++-- make-distros-flight | 22 ++++++++++++++++++++ ts-debian-di-install | 57 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 84 insertions(+), 2 deletions(-)