From patchwork Fri Sep 25 20:32:47 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 304360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E330AC4727E for ; Fri, 25 Sep 2020 20:34:36 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 39601206FC for ; Fri, 25 Sep 2020 20:34:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FO/Smi+E" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39601206FC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:38994 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLuQN-0002jX-Ac for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 16:34:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46538) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLuOs-0001P0-Br for qemu-devel@nongnu.org; Fri, 25 Sep 2020 16:33:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:50433) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLuOq-0004hH-6M for qemu-devel@nongnu.org; Fri, 25 Sep 2020 16:33:01 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601065979; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mPj7nn1jIF484UQtbNudZY2mWhzmnzITwVFl2WATscQ=; b=FO/Smi+E3vL3/IVLPiHHfHVL60B4dE1ibjRCeEYfDbaJEZBVzqnMZWMil2Sxw1LV1LDVNs H5tnFGAL6b5xDjZ8gmiK4VlYBk33uKHr7r8O6Qb2WDX7CVVgVI72qnurHeq7FwSy+He/RV itYgUc1q+AgME9iyjN5LfruBic1PFlc= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-500-NRyy2JCZN-21PkHZfeVlVw-1; Fri, 25 Sep 2020 16:32:57 -0400 X-MC-Unique: NRyy2JCZN-21PkHZfeVlVw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 693341DE05; Fri, 25 Sep 2020 20:32:56 +0000 (UTC) Received: from blue.redhat.com (ovpn-114-148.phx2.redhat.com [10.3.114.148]) by smtp.corp.redhat.com (Postfix) with ESMTP id B35E65C1BB; Fri, 25 Sep 2020 20:32:55 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Subject: [PATCH 1/3] nbd: Simplify meta-context parsing Date: Fri, 25 Sep 2020 15:32:47 -0500 Message-Id: <20200925203249.155705-2-eblake@redhat.com> In-Reply-To: <20200925203249.155705-1-eblake@redhat.com> References: <20200925203249.155705-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: pkrempa@redhat.com, vsementsov@virtuozzo.com, rjones@redhat.com, qemu-block@nongnu.org Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" We had a premature optimization of trying to read as little from the wire as possible while handling NBD_OPT_SET_META_CONTEXT in phases. But in reality, we HAVE to read the entire string from the client before we can get to the next command, and it is easier to just read it all at once than it is to read it in pieces. And once we do that, several functions end up no longer performing I/O, and no longer need to return a value. While simplifying this, take advantage of g_str_has_prefix for less repetition of boilerplate string length computation. Our iotests still pass; I also checked that libnbd's testsuite (which covers more corner cases of odd meta context requests) still passes. Signed-off-by: Eric Blake --- nbd/server.c | 172 ++++++++++++++------------------------------------- 1 file changed, 47 insertions(+), 125 deletions(-) diff --git a/nbd/server.c b/nbd/server.c index 982de67816a7..0d2d7e52058f 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -1,5 +1,5 @@ /* - * Copyright (C) 2016-2018 Red Hat, Inc. + * Copyright (C) 2016-2020 Red Hat, Inc. * Copyright (C) 2005 Anthony Liguori * * Network Block Device Server Side @@ -792,135 +792,64 @@ static int nbd_negotiate_send_meta_context(NBDClient *client, return qio_channel_writev_all(client->ioc, iov, 2, errp) < 0 ? -EIO : 0; } -/* Read strlen(@pattern) bytes, and set @match to true if they match @pattern. - * @match is never set to false. - * - * Return -errno on I/O error, 0 if option was completely handled by - * sending a reply about inconsistent lengths, or 1 on success. - * - * Note: return code = 1 doesn't mean that we've read exactly @pattern. - * It only means that there are no errors. + +/* + * Check @ns with @len bytes, and set @match to true if it matches @pattern, + * or if @len is 0 and the client is performing _LIST_. @match is never set + * to false. */ -static int nbd_meta_pattern(NBDClient *client, const char *pattern, bool *match, - Error **errp) +static void nbd_meta_empty_or_pattern(NBDClient *client, const char *pattern, + const char *ns, uint32_t len, + bool *match, Error **errp) { - int ret; - char *query; - size_t len = strlen(pattern); - - assert(len); - - query = g_malloc(len); - ret = nbd_opt_read(client, query, len, errp); - if (ret <= 0) { - g_free(query); - return ret; - } - - if (strncmp(query, pattern, len) == 0) { + if (len == 0) { + if (client->opt == NBD_OPT_LIST_META_CONTEXT) { + *match = true; + } + trace_nbd_negotiate_meta_query_parse("empty"); + } else if (strcmp(pattern, ns) == 0) { trace_nbd_negotiate_meta_query_parse(pattern); *match = true; } else { trace_nbd_negotiate_meta_query_skip("pattern not matched"); } - g_free(query); - - return 1; -} - -/* - * Read @len bytes, and set @match to true if they match @pattern, or if @len - * is 0 and the client is performing _LIST_. @match is never set to false. - * - * Return -errno on I/O error, 0 if option was completely handled by - * sending a reply about inconsistent lengths, or 1 on success. - * - * Note: return code = 1 doesn't mean that we've read exactly @pattern. - * It only means that there are no errors. - */ -static int nbd_meta_empty_or_pattern(NBDClient *client, const char *pattern, - uint32_t len, bool *match, Error **errp) -{ - if (len == 0) { - if (client->opt == NBD_OPT_LIST_META_CONTEXT) { - *match = true; - } - trace_nbd_negotiate_meta_query_parse("empty"); - return 1; - } - - if (len != strlen(pattern)) { - trace_nbd_negotiate_meta_query_skip("different lengths"); - return nbd_opt_skip(client, len, errp); - } - - return nbd_meta_pattern(client, pattern, match, errp); } /* nbd_meta_base_query * * Handle queries to 'base' namespace. For now, only the base:allocation - * context is available. 'len' is the amount of text remaining to be read from - * the current name, after the 'base:' portion has been stripped. - * - * Return -errno on I/O error, 0 if option was completely handled by - * sending a reply about inconsistent lengths, or 1 on success. + * context is available. @len is the length of @ns, including the 'base:' + * prefix. */ -static int nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, - uint32_t len, Error **errp) +static void nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, + const char *ns, uint32_t len, Error **errp) { - return nbd_meta_empty_or_pattern(client, "allocation", len, - &meta->base_allocation, errp); + nbd_meta_empty_or_pattern(client, "allocation", ns + 5, len - 5, + &meta->base_allocation, errp); } /* nbd_meta_bitmap_query * * Handle query to 'qemu:' namespace. - * @len is the amount of text remaining to be read from the current name, after - * the 'qemu:' portion has been stripped. - * - * Return -errno on I/O error, 0 if option was completely handled by - * sending a reply about inconsistent lengths, or 1 on success. */ -static int nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, - uint32_t len, Error **errp) + * @len is the length of @ns, including the `qemu:' prefix. + */ +static void nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, + const char *ns, uint32_t len, Error **errp) { - bool dirty_bitmap = false; - size_t dirty_bitmap_len = strlen("dirty-bitmap:"); - int ret; - if (!meta->exp->export_bitmap) { trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported"); - return nbd_opt_skip(client, len, errp); - } - - if (len == 0) { + } else if (len == 0) { if (client->opt == NBD_OPT_LIST_META_CONTEXT) { meta->bitmap = true; } trace_nbd_negotiate_meta_query_parse("empty"); - return 1; - } - - if (len < dirty_bitmap_len) { - trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:"); - return nbd_opt_skip(client, len, errp); - } - - len -= dirty_bitmap_len; - ret = nbd_meta_pattern(client, "dirty-bitmap:", &dirty_bitmap, errp); - if (ret <= 0) { - return ret; - } - if (!dirty_bitmap) { + } else if (!g_str_has_prefix(ns + 5, "dirty-bitmap:")) { trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:"); - return nbd_opt_skip(client, len, errp); + } else { + trace_nbd_negotiate_meta_query_parse("dirty-bitmap:"); + nbd_meta_empty_or_pattern(client, meta->exp->export_bitmap_context, + ns, len, &meta->bitmap, errp); } - - trace_nbd_negotiate_meta_query_parse("dirty-bitmap:"); - - return nbd_meta_empty_or_pattern( - client, meta->exp->export_bitmap_context + - strlen("qemu:dirty_bitmap:"), len, &meta->bitmap, errp); } /* nbd_negotiate_meta_query @@ -930,22 +859,13 @@ static int nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, * * The only supported namespaces are 'base' and 'qemu'. * - * The function aims not wasting time and memory to read long unknown namespace - * names. - * * Return -errno on I/O error, 0 if option was completely handled by * sending a reply about inconsistent lengths, or 1 on success. */ static int nbd_negotiate_meta_query(NBDClient *client, NBDExportMetaContexts *meta, Error **errp) { - /* - * Both 'qemu' and 'base' namespaces have length = 5 including a - * colon. If another length namespace is later introduced, this - * should certainly be refactored. - */ int ret; - size_t ns_len = 5; - char ns[5]; + g_autofree char *ns = NULL; uint32_t len; ret = nbd_opt_read(client, &len, sizeof(len), errp); @@ -958,27 +878,29 @@ static int nbd_negotiate_meta_query(NBDClient *client, trace_nbd_negotiate_meta_query_skip("length too long"); return nbd_opt_skip(client, len, errp); } - if (len < ns_len) { - trace_nbd_negotiate_meta_query_skip("length too short"); - return nbd_opt_skip(client, len, errp); - } - len -= ns_len; - ret = nbd_opt_read(client, ns, ns_len, errp); + ns = g_malloc(len + 1); + ret = nbd_opt_read(client, ns, len, errp); if (ret <= 0) { return ret; } + ns[len] = '\0'; + if (strlen(ns) != len) { + return nbd_opt_invalid(client, errp, + "Embedded NUL in query for option %s", + nbd_opt_lookup(client->opt)); + } - if (!strncmp(ns, "base:", ns_len)) { + if (g_str_has_prefix(ns, "base:")) { trace_nbd_negotiate_meta_query_parse("base:"); - return nbd_meta_base_query(client, meta, len, errp); - } else if (!strncmp(ns, "qemu:", ns_len)) { + nbd_meta_base_query(client, meta, ns, len, errp); + } else if (g_str_has_prefix(ns, "qemu:")) { trace_nbd_negotiate_meta_query_parse("qemu:"); - return nbd_meta_qemu_query(client, meta, len, errp); + nbd_meta_qemu_query(client, meta, ns, len, errp); + } else { + trace_nbd_negotiate_meta_query_skip("unknown namespace"); } - - trace_nbd_negotiate_meta_query_skip("unknown namespace"); - return nbd_opt_skip(client, len, errp); + return 1; } /* nbd_negotiate_meta_queries From patchwork Fri Sep 25 20:32:48 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 272681 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A94DC4727E for ; Fri, 25 Sep 2020 20:35:52 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7CA23206D4 for ; Fri, 25 Sep 2020 20:35:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Fhl22Qwy" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7CA23206D4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:43932 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLuRa-0004o4-FL for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 16:35:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46614) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLuOy-0001TZ-3J for qemu-devel@nongnu.org; Fri, 25 Sep 2020 16:33:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:40583) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLuOq-0004hS-SB for qemu-devel@nongnu.org; Fri, 25 Sep 2020 16:33:07 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601065980; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b6dTAonjWxzR83NZn2BtpM0/KnZYRcGuq3FgZZlbKKk=; b=Fhl22Qwyf6taBV1Sue7i++qajUehoHovdhiBpIo4GN1MEgsPt+Q83crNeT8cex30yrDX3p lj170sR5zinsacDN9a6t/SAgV9CbR5pS2nBcdV3k8u6j0tqAmOzvfK1dbx4ZUferMOOQtP dAI16hoXxTL0y4FnpF28JOZdNWcVxMU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-470-_wP7eicKPu2LINO3kXM2nQ-1; Fri, 25 Sep 2020 16:32:58 -0400 X-MC-Unique: _wP7eicKPu2LINO3kXM2nQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 484DF186DD42; Fri, 25 Sep 2020 20:32:57 +0000 (UTC) Received: from blue.redhat.com (ovpn-114-148.phx2.redhat.com [10.3.114.148]) by smtp.corp.redhat.com (Postfix) with ESMTP id A07715C1BB; Fri, 25 Sep 2020 20:32:56 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Subject: [PATCH 2/3] nbd: Add new qemu:allocation-depth metacontext Date: Fri, 25 Sep 2020 15:32:48 -0500 Message-Id: <20200925203249.155705-3-eblake@redhat.com> In-Reply-To: <20200925203249.155705-1-eblake@redhat.com> References: <20200925203249.155705-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , pkrempa@redhat.com, qemu-block@nongnu.org, Markus Armbruster , rjones@redhat.com, Max Reitz , vsementsov@virtuozzo.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" 'qemu-img map' provides a way to determine which extents of an image come from the top layer vs. inherited from a backing chain. This is useful information worth exposing over NBD. There is a proposal to add a QMP command block-dirty-bitmap-populate which can create a dirty bitmap that reflects allocation information, at which point qemu:dirty-bitmap:NAME can expose that information via the creation of a temporary bitmap, but we can shorten the effort by adding a new qemu:allocation-depth context that does the same thing without an intermediate bitmap (this patch does not eliminate the need for that proposal, as it will have other uses as well). For this patch, I just encoded a tri-state value (unallocated, from this layer, from any of the backing layers); we could instead or in addition report an actual depth count per extent, if that proves more useful. Note that this patch does not actually enable any way to request a server to enable this context; that will come in the next patch. Signed-off-by: Eric Blake --- docs/interop/nbd.txt | 22 +++++++-- qapi/block-core.json | 6 ++- include/block/nbd.h | 8 +++- nbd/server.c | 110 +++++++++++++++++++++++++++++++++++++++---- 4 files changed, 132 insertions(+), 14 deletions(-) diff --git a/docs/interop/nbd.txt b/docs/interop/nbd.txt index f3b3cacc9621..56efec7aee12 100644 --- a/docs/interop/nbd.txt +++ b/docs/interop/nbd.txt @@ -17,9 +17,9 @@ namespace "qemu". == "qemu" namespace == -The "qemu" namespace currently contains only one type of context, -related to exposing the contents of a dirty bitmap alongside the -associated disk contents. That context has the following form: +The "qemu" namespace currently contains two types of context. The +first is related to exposing the contents of a dirty bitmap alongside +the associated disk contents. That context has the following form: qemu:dirty-bitmap: @@ -28,8 +28,21 @@ in reply for NBD_CMD_BLOCK_STATUS: bit 0: NBD_STATE_DIRTY, means that the extent is "dirty" +The second is related to exposing the source of various extents within +the image, with a single context named: + + qemu:allocation-depth + +In the allocation depth context, bits 0 and 1 form a tri-state value: + + bits 0-1 clear: NBD_STATE_DEPTH_UNALLOC, means the extent is unallocated + bit 0 set: NBD_STATE_DEPTH_LOCAL, the extent is allocated in this image + bit 1 set: NBD_STATE_DEPTH_BACKING, the extent is inherited from a + backing layer + For NBD_OPT_LIST_META_CONTEXT the following queries are supported -in addition to "qemu:dirty-bitmap:": +in addition to the specific "qemu:allocation-depth" and +"qemu:dirty-bitmap:": * "qemu:" - returns list of all available metadata contexts in the namespace. @@ -55,3 +68,4 @@ the operation of that feature. NBD_CMD_BLOCK_STATUS for "qemu:dirty-bitmap:", NBD_CMD_CACHE * 4.2: NBD_FLAG_CAN_MULTI_CONN for shareable read-only exports, NBD_CMD_FLAG_FAST_ZERO +* 5.2: NBD_CMD_BLOCK_STATUS for "qemu:allocation-depth" diff --git a/qapi/block-core.json b/qapi/block-core.json index 3c16f1e11d6b..b3ec30a83cd7 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -5261,11 +5261,15 @@ # NBD client can use NBD_OPT_SET_META_CONTEXT with # "qemu:dirty-bitmap:NAME" to inspect the bitmap. (since 4.0) # +# @alloc: Also export the allocation map for @device, so the NBD client +# can use NBD_OPT_SET_META_CONTEXT with "qemu:allocation-depth" +# to inspect allocation details. (since 5.2) +# # Since: 5.0 ## { 'struct': 'BlockExportNbd', 'data': {'device': 'str', '*name': 'str', '*description': 'str', - '*writable': 'bool', '*bitmap': 'str' } } + '*writable': 'bool', '*bitmap': 'str', '*alloc': 'bool' } } ## # @nbd-server-add: diff --git a/include/block/nbd.h b/include/block/nbd.h index 9bc3bfaeecf8..71a2623f7842 100644 --- a/include/block/nbd.h +++ b/include/block/nbd.h @@ -1,5 +1,5 @@ /* - * Copyright (C) 2016-2019 Red Hat, Inc. + * Copyright (C) 2016-2020 Red Hat, Inc. * Copyright (C) 2005 Anthony Liguori * * Network Block Device @@ -257,6 +257,12 @@ enum { /* Extent flags for qemu:dirty-bitmap in NBD_REPLY_TYPE_BLOCK_STATUS */ #define NBD_STATE_DIRTY (1 << 0) +/* Extent flags for qemu:allocation-depth in NBD_REPLY_TYPE_BLOCK_STATUS */ +#define NBD_STATE_DEPTH_UNALLOC (0 << 0) +#define NBD_STATE_DEPTH_LOCAL (1 << 0) +#define NBD_STATE_DEPTH_BACKING (2 << 0) +#define NBD_STATE_DEPTH_MASK (0x3) + static inline bool nbd_reply_type_is_error(int type) { return type & (1 << 15); diff --git a/nbd/server.c b/nbd/server.c index 0d2d7e52058f..02d5fb375b24 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -25,7 +25,8 @@ #include "qemu/units.h" #define NBD_META_ID_BASE_ALLOCATION 0 -#define NBD_META_ID_DIRTY_BITMAP 1 +#define NBD_META_ID_ALLOCATION_DEPTH 1 +#define NBD_META_ID_DIRTY_BITMAP 2 /* * NBD_MAX_BLOCK_STATUS_EXTENTS: 1 MiB of extents data. An empirical @@ -97,6 +98,7 @@ struct NBDExport { BlockBackend *eject_notifier_blk; Notifier eject_notifier; + bool alloc_context; BdrvDirtyBitmap *export_bitmap; char *export_bitmap_context; }; @@ -113,6 +115,7 @@ typedef struct NBDExportMetaContexts { bool valid; /* means that negotiation of the option finished without errors */ bool base_allocation; /* export base:allocation context (block status) */ + bool allocation_depth; /* export qemu:allocation-depth */ bool bitmap; /* export qemu:dirty-bitmap: */ } NBDExportMetaContexts; @@ -836,15 +839,24 @@ static void nbd_meta_base_query(NBDClient *client, NBDExportMetaContexts *meta, static void nbd_meta_qemu_query(NBDClient *client, NBDExportMetaContexts *meta, const char *ns, uint32_t len, Error **errp) { - if (!meta->exp->export_bitmap) { - trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported"); - } else if (len == 0) { + if (len == 0) { if (client->opt == NBD_OPT_LIST_META_CONTEXT) { - meta->bitmap = true; + meta->allocation_depth = meta->exp->alloc_context; + meta->bitmap = !!meta->exp->export_bitmap; } trace_nbd_negotiate_meta_query_parse("empty"); + } else if (strcmp(ns + 5, "allocation-depth") == 0) { + if (meta->exp->alloc_context) { + trace_nbd_negotiate_meta_query_parse("allocation-depth"); + meta->allocation_depth = true; + } else { + trace_nbd_negotiate_meta_query_skip("allocation-depth not " + "available"); + } } else if (!g_str_has_prefix(ns + 5, "dirty-bitmap:")) { trace_nbd_negotiate_meta_query_skip("not dirty-bitmap:"); + } else if (!meta->exp->export_bitmap) { + trace_nbd_negotiate_meta_query_skip("no dirty-bitmap exported"); } else { trace_nbd_negotiate_meta_query_parse("dirty-bitmap:"); nbd_meta_empty_or_pattern(client, meta->exp->export_bitmap_context, @@ -954,6 +966,7 @@ static int nbd_negotiate_meta_queries(NBDClient *client, if (client->opt == NBD_OPT_LIST_META_CONTEXT && !nb_queries) { /* enable all known contexts */ meta->base_allocation = true; + meta->allocation_depth = meta->exp->alloc_context; meta->bitmap = !!meta->exp->export_bitmap; } else { for (i = 0; i < nb_queries; ++i) { @@ -973,6 +986,15 @@ static int nbd_negotiate_meta_queries(NBDClient *client, } } + if (meta->allocation_depth) { + ret = nbd_negotiate_send_meta_context(client, "qemu:allocation-depth", + NBD_META_ID_ALLOCATION_DEPTH, + errp); + if (ret < 0) { + return ret; + } + } + if (meta->bitmap) { ret = nbd_negotiate_send_meta_context(client, meta->exp->export_bitmap_context, @@ -1972,6 +1994,40 @@ static int blockstatus_to_extents(BlockDriverState *bs, uint64_t offset, return 0; } +static int blockalloc_to_extents(BlockDriverState *bs, uint64_t offset, + uint64_t bytes, NBDExtentArray *ea) +{ + while (bytes) { + uint32_t flags; + int64_t num; + int ret = bdrv_is_allocated(bs, offset, bytes, &num); + + if (ret < 0) { + return ret; + } + + if (ret == 1) { + flags = NBD_STATE_DEPTH_LOCAL; + } else { + ret = bdrv_is_allocated_above(bs, NULL, false, offset, num, + &num); + if (ret < 0) { + return ret; + } + flags = ret ? NBD_STATE_DEPTH_BACKING : NBD_STATE_DEPTH_UNALLOC; + } + + if (nbd_extent_array_add(ea, num, flags) < 0) { + return 0; + } + + offset += num; + bytes -= num; + } + + return 0; +} + /* * nbd_co_send_extents * @@ -2020,6 +2076,26 @@ static int nbd_co_send_block_status(NBDClient *client, uint64_t handle, return nbd_co_send_extents(client, handle, ea, last, context_id, errp); } +/* Get allocation depth from the exported device and send it to the client */ +static int nbd_co_send_block_alloc(NBDClient *client, uint64_t handle, + BlockDriverState *bs, uint64_t offset, + uint32_t length, bool dont_fragment, + bool last, uint32_t context_id, + Error **errp) +{ + int ret; + unsigned int nb_extents = dont_fragment ? 1 : NBD_MAX_BLOCK_STATUS_EXTENTS; + g_autoptr(NBDExtentArray) ea = nbd_extent_array_new(nb_extents); + + ret = blockalloc_to_extents(bs, offset, length, ea); + if (ret < 0) { + return nbd_co_send_structured_error( + client, handle, -ret, "can't get block status", errp); + } + + return nbd_co_send_extents(client, handle, ea, last, context_id, errp); +} + /* Populate @ea from a dirty bitmap. */ static void bitmap_to_extents(BdrvDirtyBitmap *bitmap, uint64_t offset, uint64_t length, @@ -2348,15 +2424,19 @@ static coroutine_fn int nbd_handle_request(NBDClient *client, } if (client->export_meta.valid && (client->export_meta.base_allocation || + client->export_meta.allocation_depth || client->export_meta.bitmap)) { bool dont_fragment = request->flags & NBD_CMD_FLAG_REQ_ONE; + int contexts_remaining = client->export_meta.base_allocation + + client->export_meta.allocation_depth + + client->export_meta.bitmap; if (client->export_meta.base_allocation) { ret = nbd_co_send_block_status(client, request->handle, blk_bs(exp->blk), request->from, request->len, dont_fragment, - !client->export_meta.bitmap, + !--contexts_remaining, NBD_META_ID_BASE_ALLOCATION, errp); if (ret < 0) { @@ -2364,17 +2444,31 @@ static coroutine_fn int nbd_handle_request(NBDClient *client, } } + if (client->export_meta.allocation_depth) { + ret = nbd_co_send_block_alloc(client, request->handle, + blk_bs(exp->blk), request->from, + request->len, dont_fragment, + !--contexts_remaining, + NBD_META_ID_ALLOCATION_DEPTH, + errp); + if (ret < 0) { + return ret; + } + } + if (client->export_meta.bitmap) { ret = nbd_co_send_bitmap(client, request->handle, client->exp->export_bitmap, request->from, request->len, - dont_fragment, - true, NBD_META_ID_DIRTY_BITMAP, errp); + dont_fragment, !--contexts_remaining, + NBD_META_ID_DIRTY_BITMAP, errp); if (ret < 0) { return ret; } } + assert(!contexts_remaining); + return 0; } else { return nbd_send_generic_reply(client, request->handle, -EINVAL, From patchwork Fri Sep 25 20:32:49 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Eric Blake X-Patchwork-Id: 272682 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B6503C4727E for ; Fri, 25 Sep 2020 20:34:50 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E0141206FC for ; Fri, 25 Sep 2020 20:34:49 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gFRk6MuD" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E0141206FC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:39610 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kLuQa-0002yY-NK for qemu-devel@archiver.kernel.org; Fri, 25 Sep 2020 16:34:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46616) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kLuOz-0001Vz-0J for qemu-devel@nongnu.org; Fri, 25 Sep 2020 16:33:09 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:56511) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1kLuOw-0004hl-6Y for qemu-devel@nongnu.org; Fri, 25 Sep 2020 16:33:08 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601065982; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IsJvNJS7L2JYP6gKta1dJI8uaoFwaEjNa7oZL6/WB0Y=; b=gFRk6MuD17N5a/aSNWD1D76sIJa59LF5g/lSv02zDjPUJeM/3qB0UNjuHUm20Q47mFTDGk yYDtlo9YVJrnhkvzsonW/2vF5LHmAKRyPGrtkMzZj9KafS86213LZ5MYsFBapx8N20Dq8Q 7X5uKYImNdVpuVeE3Epv5dnK8L+eiSY= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-2-DqkBF0GMNAOG_FJom5bodg-1; Fri, 25 Sep 2020 16:32:59 -0400 X-MC-Unique: DqkBF0GMNAOG_FJom5bodg-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2CC8480732A; Fri, 25 Sep 2020 20:32:58 +0000 (UTC) Received: from blue.redhat.com (ovpn-114-148.phx2.redhat.com [10.3.114.148]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8735B5C1BB; Fri, 25 Sep 2020 20:32:57 +0000 (UTC) From: Eric Blake To: qemu-devel@nongnu.org Subject: [PATCH 3/3] nbd: Add 'qemu-nbd -A' to expose allocation depth Date: Fri, 25 Sep 2020 15:32:49 -0500 Message-Id: <20200925203249.155705-4-eblake@redhat.com> In-Reply-To: <20200925203249.155705-1-eblake@redhat.com> References: <20200925203249.155705-1-eblake@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eblake@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Received-SPF: pass client-ip=63.128.21.124; envelope-from=eblake@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/25 01:07:33 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -32 X-Spam_score: -3.3 X-Spam_bar: --- X-Spam_report: (-3.3 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1.199, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , pkrempa@redhat.com, qemu-block@nongnu.org, Markus Armbruster , rjones@redhat.com, Max Reitz , vsementsov@virtuozzo.com Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Allow the server to expose an additional metacontext to be requested by savvy clients. qemu-nbd adds a new option -A to expose the qemu:allocation-depth metacontext through NBD_CMD_BLOCK_STATUS; this can also be set via QMP when using nbd-server-add. qemu as client can be hacked into viewing this new context by using the now-misnamed x-dirty-bitmap option when creating an NBD blockdev; although it is worth noting the decoding of how such context information will appear in 'qemu-img map --output=json': NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false NBD_STATE_DEPTH_BACKING => "zero":true, "data":true libnbd as client is probably a nicer way to get at the information without having to decipher such hacks in qemu as client. ;) Signed-off-by: Eric Blake --- docs/tools/qemu-nbd.rst | 6 ++++ qapi/block-core.json | 7 ++-- include/block/nbd.h | 7 ++-- blockdev-nbd.c | 3 +- nbd/server.c | 8 +++-- qemu-nbd.c | 16 ++++++--- tests/qemu-iotests/309 | 73 ++++++++++++++++++++++++++++++++++++++ tests/qemu-iotests/309.out | 22 ++++++++++++ tests/qemu-iotests/group | 1 + 9 files changed, 129 insertions(+), 14 deletions(-) create mode 100755 tests/qemu-iotests/309 create mode 100644 tests/qemu-iotests/309.out diff --git a/docs/tools/qemu-nbd.rst b/docs/tools/qemu-nbd.rst index 667861cb22e9..0e545a97cfa3 100644 --- a/docs/tools/qemu-nbd.rst +++ b/docs/tools/qemu-nbd.rst @@ -72,6 +72,12 @@ driver options if ``--image-opts`` is specified. Export the disk as read-only. +.. option:: -A, --allocation-depth + + Expose allocation depth information via the + ``qemu:allocation-depth`` context accessible through + NBD_OPT_SET_META_CONTEXT. + .. option:: -B, --bitmap=NAME If *filename* has a qcow2 persistent bitmap *NAME*, expose diff --git a/qapi/block-core.json b/qapi/block-core.json index b3ec30a83cd7..84f7a7a34dc1 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json @@ -3874,9 +3874,12 @@ # # @tls-creds: TLS credentials ID # -# @x-dirty-bitmap: A "qemu:dirty-bitmap:NAME" string to query in place of +# @x-dirty-bitmap: A metacontext name such as "qemu:dirty-bitmap:NAME" or +# "qemu:allocation-depth" to query in place of the # traditional "base:allocation" block status (see -# NBD_OPT_LIST_META_CONTEXT in the NBD protocol) (since 3.0) +# NBD_OPT_LIST_META_CONTEXT in the NBD protocol; and +# yes, naming this option x-context would have made +# more sense) (since 3.0) # # @reconnect-delay: On an unexpected disconnect, the nbd client tries to # connect again until succeeding or encountering a serious diff --git a/include/block/nbd.h b/include/block/nbd.h index 71a2623f7842..f660e0274ce3 100644 --- a/include/block/nbd.h +++ b/include/block/nbd.h @@ -336,9 +336,10 @@ typedef struct NBDClient NBDClient; NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset, uint64_t size, const char *name, const char *desc, - const char *bitmap, bool readonly, bool shared, - void (*close)(NBDExport *), bool writethrough, - BlockBackend *on_eject_blk, Error **errp); + bool alloc_depth, const char *bitmap, bool readonly, + bool shared, void (*close)(NBDExport *), + bool writethrough, BlockBackend *on_eject_blk, + Error **errp); void nbd_export_close(NBDExport *exp); void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp); void nbd_export_get(NBDExport *exp); diff --git a/blockdev-nbd.c b/blockdev-nbd.c index 1a95d89f0096..562ea3751b85 100644 --- a/blockdev-nbd.c +++ b/blockdev-nbd.c @@ -203,7 +203,8 @@ void qmp_nbd_server_add(BlockExportNbd *arg, Error **errp) arg->writable = false; } - exp = nbd_export_new(bs, 0, len, arg->name, arg->description, arg->bitmap, + exp = nbd_export_new(bs, 0, len, arg->name, arg->description, + arg->alloc, arg->bitmap, !arg->writable, !arg->writable, NULL, false, on_eject_blk, errp); if (!exp) { diff --git a/nbd/server.c b/nbd/server.c index 02d5fb375b24..f5e0e115b703 100644 --- a/nbd/server.c +++ b/nbd/server.c @@ -1450,9 +1450,10 @@ static void nbd_eject_notifier(Notifier *n, void *data) NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset, uint64_t size, const char *name, const char *desc, - const char *bitmap, bool readonly, bool shared, - void (*close)(NBDExport *), bool writethrough, - BlockBackend *on_eject_blk, Error **errp) + bool alloc_depth, const char *bitmap, bool readonly, + bool shared, void (*close)(NBDExport *), + bool writethrough, BlockBackend *on_eject_blk, + Error **errp) { AioContext *ctx; BlockBackend *blk; @@ -1545,6 +1546,7 @@ NBDExport *nbd_export_new(BlockDriverState *bs, uint64_t dev_offset, assert(strlen(exp->export_bitmap_context) < NBD_MAX_STRING_SIZE); } + exp->alloc_context = alloc_depth; exp->close = close; exp->ctx = ctx; blk_add_aio_context_notifier(blk, blk_aio_attached, blk_aio_detach, exp); diff --git a/qemu-nbd.c b/qemu-nbd.c index e39d3b23c121..5aeef60b0a5a 100644 --- a/qemu-nbd.c +++ b/qemu-nbd.c @@ -100,6 +100,7 @@ static void usage(const char *name) "\n" "Exposing part of the image:\n" " -o, --offset=OFFSET offset into the image\n" +" -A, --allocation-depth expose the allocation depth\n" " -B, --bitmap=NAME expose a persistent dirty bitmap\n" "\n" "General purpose options:\n" @@ -527,7 +528,7 @@ int main(int argc, char **argv) int64_t fd_size; QemuOpts *sn_opts = NULL; const char *sn_id_or_name = NULL; - const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:B:L"; + const char *sopt = "hVb:o:p:rsnc:dvk:e:f:tl:x:T:D:AB:L"; struct option lopt[] = { { "help", no_argument, NULL, 'h' }, { "version", no_argument, NULL, 'V' }, @@ -536,6 +537,7 @@ int main(int argc, char **argv) { "socket", required_argument, NULL, 'k' }, { "offset", required_argument, NULL, 'o' }, { "read-only", no_argument, NULL, 'r' }, + { "allocation-depth", no_argument, NULL, 'A' }, { "bitmap", required_argument, NULL, 'B' }, { "connect", required_argument, NULL, 'c' }, { "disconnect", no_argument, NULL, 'd' }, @@ -577,6 +579,7 @@ int main(int argc, char **argv) QDict *options = NULL; const char *export_name = NULL; /* defaults to "" later for server mode */ const char *export_description = NULL; + bool alloc_depth = false; const char *bitmap = NULL; const char *tlscredsid = NULL; bool imageOpts = false; @@ -700,6 +703,9 @@ int main(int argc, char **argv) readonly = true; flags &= ~BDRV_O_RDWR; break; + case 'A': + alloc_depth = true; + break; case 'B': bitmap = optarg; break; @@ -797,8 +803,8 @@ int main(int argc, char **argv) exit(EXIT_FAILURE); } if (export_name || export_description || dev_offset || - device || disconnect || fmt || sn_id_or_name || bitmap || - seen_aio || seen_discard || seen_cache) { + device || disconnect || fmt || sn_id_or_name || alloc_depth || + bitmap || seen_aio || seen_discard || seen_cache) { error_report("List mode is incompatible with per-device settings"); exit(EXIT_FAILURE); } @@ -1069,8 +1075,8 @@ int main(int argc, char **argv) fd_size -= dev_offset; export = nbd_export_new(bs, dev_offset, fd_size, export_name, - export_description, bitmap, readonly, shared > 1, - nbd_export_closed, writethrough, NULL, + export_description, alloc_depth, bitmap, readonly, + shared > 1, nbd_export_closed, writethrough, NULL, &error_fatal); if (device) { diff --git a/tests/qemu-iotests/309 b/tests/qemu-iotests/309 new file mode 100755 index 000000000000..fda3be9abbd2 --- /dev/null +++ b/tests/qemu-iotests/309 @@ -0,0 +1,73 @@ +#!/usr/bin/env bash +# +# Test qemu-nbd -A +# +# Copyright (C) 2018-2020 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# + +seq="$(basename $0)" +echo "QA output created by $seq" + +status=1 # failure is the default! + +_cleanup() +{ + _cleanup_test_img + nbd_server_stop +} +trap "_cleanup; exit \$status" 0 1 2 3 15 + +# get standard environment, filters and checks +. ./common.rc +. ./common.filter +. ./common.nbd + +_supported_fmt qcow2 +_supported_proto file +_supported_os Linux +_require_command QEMU_NBD + +echo +echo "=== Initial image setup ===" +echo + +TEST_IMG="$TEST_IMG.base" _make_test_img 4M +$QEMU_IO -c 'w 0 2M' -f $IMGFMT "$TEST_IMG.base" | _filter_qemu_io +_make_test_img -b "$TEST_IMG.base" -F $IMGFMT 4M +$QEMU_IO -c 'w 1M 2M' -f $IMGFMT "$TEST_IMG" | _filter_qemu_io + +echo +echo "=== Check allocation over NBD ===" +echo + +# Normal -f raw NBD block status loses access to allocation information +$QEMU_IMG map --output=json -f qcow2 "$TEST_IMG" +IMG="driver=nbd,server.type=unix,server.path=$nbd_unix_socket" +nbd_server_start_unix_socket -r -f qcow2 -A "$TEST_IMG" +$QEMU_IMG map --output=json --image-opts \ + "$IMG" | _filter_qemu_img_map +# But since we used -A, and use x-dirty-bitmap as a hack for reading bitmaps, +# we can reconstruct it, by abusing block status to report: +# NBD_STATE_DEPTH_UNALLOC => "zero":false, "data":true +# NBD_STATE_DEPTH_LOCAL => "zero":false, "data":false +# NBD_STATE_DEPTH_BACKING => "zero":true, "data":true +$QEMU_IMG map --output=json --image-opts \ + "$IMG,x-dirty-bitmap=qemu:allocation-depth" | _filter_qemu_img_map + +# success, all done +echo '*** done' +rm -f $seq.full +status=0 diff --git a/tests/qemu-iotests/309.out b/tests/qemu-iotests/309.out new file mode 100644 index 000000000000..db75bb6b0df9 --- /dev/null +++ b/tests/qemu-iotests/309.out @@ -0,0 +1,22 @@ +QA output created by 309 + +=== Initial image setup === + +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=4194304 +wrote 2097152/2097152 bytes at offset 0 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4194304 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT +wrote 2097152/2097152 bytes at offset 1048576 +2 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec) + +=== Check allocation over NBD === + +[{ "start": 0, "length": 1048576, "depth": 1, "zero": false, "data": true, "offset": 327680}, +{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": true, "offset": 327680}, +{ "start": 3145728, "length": 1048576, "depth": 1, "zero": true, "data": false}] +[{ "start": 0, "length": 3145728, "depth": 0, "zero": false, "data": true, "offset": OFFSET}, +{ "start": 3145728, "length": 1048576, "depth": 0, "zero": true, "data": false, "offset": OFFSET}] +[{ "start": 0, "length": 1048576, "depth": 0, "zero": true, "data": true, "offset": OFFSET}, +{ "start": 1048576, "length": 2097152, "depth": 0, "zero": false, "data": false}, +{ "start": 3145728, "length": 1048576, "depth": 0, "zero": false, "data": true, "offset": OFFSET}] +*** done diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group index ff59cfd2d4c4..57182e4b3169 100644 --- a/tests/qemu-iotests/group +++ b/tests/qemu-iotests/group @@ -314,3 +314,4 @@ 303 rw quick 304 rw quick 305 rw quick +309 rw auto quick