From patchwork Mon May 4 12:49:58 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Auger Eric X-Patchwork-Id: 48004 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wg0-f71.google.com (mail-wg0-f71.google.com [74.125.82.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 272162121F for ; Mon, 4 May 2015 12:50:18 +0000 (UTC) Received: by wgtl5 with SMTP id l5sf44216586wgt.1 for ; Mon, 04 May 2015 05:50:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe; bh=dLk6ib5T19uikYL5FlvX3bfL4HNaf0gRdsgDgnXPhfo=; b=jeR6ZbBZh9loHuZdhIPfgboV7IDBdarve29xrYwuicIDGDsMaIpSnoQKTrX3xQRmc6 I8OoaLPQyZ5bfyIzt5CiophfiyqMXwjJPLzfvjpPxd1MO+SzzXymOggGvZPj98mldd/d 9hHYuUvZ196fLWs1lKh6D59UDRMK3gwUGH8gfpGQ9F7wDTdNbNiugCGJhtzH7zNhG/uE 2Dejjr+0VMtTjZ2v/fwcXT15W6KnjwzXuuqjds6gDyowTsOjqD19odr9CsT/+D0m5Wzh 23eMDe1pz80hfwk8EXdv4V3aX2P4NbXY+dh76OrkPl4VydT/1e+e0Yu+TbOlM391HOxY 8Crw== X-Gm-Message-State: ALoCoQnM5G72Wemg/Svgg68XlZZjeSwLLB2XlGu4n9ycKI/MJ4WiZZWVU+Qflwk5lK0Gmle7kE/Y X-Received: by 10.180.96.6 with SMTP id do6mr8357212wib.4.1430743817488; Mon, 04 May 2015 05:50:17 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.23.232 with SMTP id p8ls749358laf.13.gmail; Mon, 04 May 2015 05:50:17 -0700 (PDT) X-Received: by 10.152.238.43 with SMTP id vh11mr19522148lac.45.1430743817288; Mon, 04 May 2015 05:50:17 -0700 (PDT) Received: from mail-la0-f50.google.com (mail-la0-f50.google.com. [209.85.215.50]) by mx.google.com with ESMTPS id yd8si10025431lbb.82.2015.05.04.05.50.17 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 May 2015 05:50:17 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) client-ip=209.85.215.50; Received: by layy10 with SMTP id y10so103237552lay.0 for ; Mon, 04 May 2015 05:50:17 -0700 (PDT) X-Received: by 10.112.219.70 with SMTP id pm6mr19085372lbc.41.1430743817177; Mon, 04 May 2015 05:50:17 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.112.67.65 with SMTP id l1csp1641128lbt; Mon, 4 May 2015 05:50:16 -0700 (PDT) X-Received: by 10.194.173.226 with SMTP id bn2mr41853753wjc.148.1430743810749; Mon, 04 May 2015 05:50:10 -0700 (PDT) Received: from mail-wg0-f54.google.com (mail-wg0-f54.google.com. [74.125.82.54]) by mx.google.com with ESMTPS id gn6si22513106wjc.169.2015.05.04.05.50.10 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 May 2015 05:50:10 -0700 (PDT) Received-SPF: pass (google.com: domain of eric.auger@linaro.org designates 74.125.82.54 as permitted sender) client-ip=74.125.82.54; Received: by wgyo15 with SMTP id o15so148984598wgy.2 for ; Mon, 04 May 2015 05:50:10 -0700 (PDT) X-Received: by 10.180.105.74 with SMTP id gk10mr19309271wib.29.1430743810223; Mon, 04 May 2015 05:50:10 -0700 (PDT) Received: from midway01-04-00.lavalab ([81.128.185.50]) by mx.google.com with ESMTPSA id z13sm20482370wjr.44.2015.05.04.05.50.08 (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 04 May 2015 05:50:09 -0700 (PDT) From: Eric Auger To: eric.auger@st.com, eric.auger@linaro.org, qemu-devel@nongnu.org, alex.williamson@redhat.com Cc: christoffer.dall@linaro.org, kvmarm@lists.cs.columbia.edu, patches@linaro.org, b.reynal@virtualopensystems.com Subject: [RFC v2 4/4] hw/vfio/platform: add forwarded irq support Date: Mon, 4 May 2015 13:49:58 +0100 Message-Id: <1430743798-8839-5-git-send-email-eric.auger@linaro.org> X-Mailer: git-send-email 1.8.3.2 In-Reply-To: <1430743798-8839-1-git-send-email-eric.auger@linaro.org> References: <1430743798-8839-1-git-send-email-eric.auger@linaro.org> X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: eric.auger@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.50 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Tests whether the forwarded IRQ modality is available. In the positive device IRQs are forwarded. This control is achieved with KVM-VFIO device. with such a modality injection still is handled through irqfds. However end of interrupt is not trapped anymore. As soon as the guest completes its virtual IRQ, the corresponding physical IRQ is completed and the same physical IRQ can hit again. A new x-forward property enables to force forwarding off although enabled by the kernel. Signed-off-by: Eric Auger --- v1 -> v2: - use kvm_vfio_get|put_device_irq, new irq connect notifier and integrate with 2 stage eventfd/irqfd setup v1: - moved in a separate series v8 -> v9 (KVM platform device passthrough series): - use new kvm_vfio_dev_irq struct Signed-off-by: Eric Auger --- hw/vfio/platform.c | 103 +++++++++++++++++++++++++++++++++++++--- include/hw/vfio/vfio-platform.h | 3 ++ trace-events | 1 + 3 files changed, 100 insertions(+), 7 deletions(-) diff --git a/hw/vfio/platform.c b/hw/vfio/platform.c index 901b98e..52c6d59 100644 --- a/hw/vfio/platform.c +++ b/hw/vfio/platform.c @@ -411,6 +411,88 @@ fail_irqfd: return; } +/* + * Functions used with forwarding capability + */ + +static bool has_kvm_vfio_forward_capability(void) +{ + struct kvm_device_attr attr = { + .group = KVM_DEV_VFIO_DEVICE, + .attr = KVM_DEV_VFIO_DEVICE_FORWARD_IRQ}; + + if (ioctl(vfio_kvm_device_fd, KVM_HAS_DEVICE_ATTR, &attr) == 0) { + return true; + } else { + return false; + } +} + +static void vfio_start_forward_injection(SysBusDevice *sbdev, qemu_irq irq) +{ + VFIOPlatformDevice *vdev = VFIO_PLATFORM_DEVICE(sbdev); + VFIOINTp *intp; + bool found = false; + struct kvm_device_attr attr = { + .group = KVM_DEV_VFIO_DEVICE, + .attr = KVM_DEV_VFIO_DEVICE_FORWARD_IRQ}; + + QLIST_FOREACH(intp, &vdev->intp_list, next) { + if (intp->qemuirq == irq) { + found = true; + break; + } + } + assert(found); + + if (intp->forwarded) { + return; + } + + /* + * stop VFIO signaling and unmask the physical IRQ since + * forwarding cannot be set if the IRQ is active or vfio masked + */ + vfio_disable_irqindex(&intp->vdev->vbasedev, intp->pin); + vfio_unmask_single_irqindex(&intp->vdev->vbasedev, intp->pin); + + kvm_vfio_get_device_irq(kvm_state, intp->vdev->vbasedev.fd, + intp->pin, 0, 1, intp->qemuirq, &intp->fwd_irq); + + attr.addr = (uint64_t)(unsigned long)intp->fwd_irq; + + if (ioctl(vfio_kvm_device_fd, KVM_SET_DEVICE_ATTR, &attr) < 0) { + error_report("vfio: failed to forward irq %d, do standard irqfd", + intp->pin); + kvm_vfio_put_device_irq(intp->fwd_irq); + } else { + trace_vfio_platform_start_fwd_injection(intp->pin); + intp->forwarded = true; + } + + if (kvm_irqchip_add_irqfd_notifier(kvm_state, &intp->interrupt, + &intp->unmask, irq) < 0) { + goto fail_irqfd; + } + + if (vfio_set_trigger_eventfd(intp, NULL) < 0) { + goto fail_vfio; + } + /* only used if forwarding setup failed */ + if (vfio_set_resample_eventfd(intp) < 0) { + goto fail_vfio; + } + + intp->kvm_accel = true; + return; +fail_vfio: + kvm_irqchip_remove_irqfd_notifier(kvm_state, &intp->interrupt, irq); +fail_irqfd: + vfio_start_eventfd_injection(intp); + vfio_unmask_single_irqindex(&vdev->vbasedev, intp->pin); + return; +} + #endif /* CONFIG_KVM */ /* VFIO skeleton */ @@ -655,13 +737,6 @@ static void vfio_platform_realize(DeviceState *dev, Error **errp) vbasedev->type = VFIO_DEVICE_TYPE_PLATFORM; vbasedev->ops = &vfio_platform_ops; -#ifdef CONFIG_KVM - if (kvm_irqfds_enabled() && kvm_resamplefds_enabled() && - vdev->irqfd_allowed) { - sbc->connect_irq_notifier = vfio_start_irqfd_injection; - } -#endif - trace_vfio_platform_realize(vbasedev->name, vdev->compat); ret = vfio_base_device_init(vbasedev); @@ -679,6 +754,19 @@ static void vfio_platform_realize(DeviceState *dev, Error **errp) QLIST_FOREACH(intp, &vdev->intp_list, next) { vfio_start_eventfd_injection(intp); } + +#ifdef CONFIG_KVM + if (kvm_irqfds_enabled() && kvm_resamplefds_enabled() && + vdev->irqfd_allowed) { + if (has_kvm_vfio_forward_capability() && + vdev->forward_allowed) { + sbc->connect_irq_notifier = vfio_start_forward_injection; + } else { + sbc->connect_irq_notifier = vfio_start_irqfd_injection; + } + } +#endif + } static const VMStateDescription vfio_platform_vmstate = { @@ -692,6 +780,7 @@ static Property vfio_platform_dev_properties[] = { DEFINE_PROP_UINT32("mmap-timeout-ms", VFIOPlatformDevice, mmap_timeout, 1100), DEFINE_PROP_BOOL("x-irqfd", VFIOPlatformDevice, irqfd_allowed, true), + DEFINE_PROP_BOOL("x-forward", VFIOPlatformDevice, forward_allowed, true), DEFINE_PROP_END_OF_LIST(), }; diff --git a/include/hw/vfio/vfio-platform.h b/include/hw/vfio/vfio-platform.h index c5cf1d7..900f918 100644 --- a/include/hw/vfio/vfio-platform.h +++ b/include/hw/vfio/vfio-platform.h @@ -42,6 +42,8 @@ typedef struct VFIOINTp { uint8_t pin; /* index */ uint32_t flags; /* IRQ info flags */ bool kvm_accel; /* set when QEMU bypass through KVM enabled */ + struct kvm_vfio_dev_irq *fwd_irq; + bool forwarded; } VFIOINTp; /* function type for user side eventfd handler */ @@ -59,6 +61,7 @@ typedef struct VFIOPlatformDevice { QEMUTimer *mmap_timer; /* allows fast-path resume after IRQ hit */ QemuMutex intp_mutex; /* protect the intp_list IRQ state */ bool irqfd_allowed; /* debug option to force irqfd on/off */ + bool forward_allowed; /* debug option to force forwarding on/off */ } VFIOPlatformDevice; typedef struct VFIOPlatformDeviceClass { diff --git a/trace-events b/trace-events index cb6381a..99d2b9c 100644 --- a/trace-events +++ b/trace-events @@ -1572,6 +1572,7 @@ vfio_platform_intp_inject_pending_lockheld(int pin, int fd) "Inject pending IRQ vfio_platform_populate_interrupts(int pin, int count, int flags) "- IRQ index %d: count %d, flags=0x%x" vfio_intp_interrupt_set_pending(int index) "irq %d is set PENDING" vfio_platform_start_irqfd_injection(int index, int fd, int resamplefd) "IRQ index=%d, fd = %d, resamplefd = %d" +vfio_platform_start_fwd_injection(int pin) "forwarding set for IRQ pin %d" #hw/acpi/memory_hotplug.c mhp_acpi_invalid_slot_selected(uint32_t slot) "0x%"PRIx32