From patchwork Wed Jul 16 14:49:01 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 33732 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f72.google.com (mail-qa0-f72.google.com [209.85.216.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 4AEB620792 for ; Wed, 16 Jul 2014 14:51:49 +0000 (UTC) Received: by mail-qa0-f72.google.com with SMTP id cm18sf2543022qab.11 for ; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:message-id:from:to:date:in-reply-to :references:organization:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=nDgd1Y7rwo1+cDpTBcDym0YnjTUS5LYasZW39bVj6CU=; b=ZOEm65r4Yj6kewvwDA9pp0C8GvceeCgh86KDoQhZXZcgCYLiRiPbrp/SSY4uXHz2VC lJze/NKNe/teCQaBOLnWle/GO3JVf3rNE5u5z8qLKJHWang7GKAuYiZWk+eRV/6paSgL RUYTKFlW1GmkwymD6cSgYEkxTTus8WoV4pKzi7NZ4Nzid4d7ztSizO3KkIDE5NXqB0f+ RRxKlC908+IMbW5m0Nxy6Y6hE+b9piQzQpZtLoyr5pumH5pN09bcvEvCeJNGirocCkqa pNaxiNh3FpwMEL2BuK8WrPKdUkeV9h9d/mq6gw1RxqkcpnV6IZwUm8KtVNVDm7kmdWuA Kg2g== X-Gm-Message-State: ALoCoQmri2LgQggFP8oe0oWyR/qZqPyk6Mf9aPE5Z59hiGCkXCodYLPqFlyAO2kHA+ZLqNZTPBVE X-Received: by 10.236.129.2 with SMTP id g2mr3983802yhi.2.1405522308935; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.29.133 with SMTP id b5ls25060qgb.43.gmail; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) X-Received: by 10.220.73.137 with SMTP id q9mr8380623vcj.64.1405522308781; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) Received: from mail-vc0-f182.google.com (mail-vc0-f182.google.com [209.85.220.182]) by mx.google.com with ESMTPS id k8si8669859vdv.58.2014.07.16.07.51.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 16 Jul 2014 07:51:48 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.182 as permitted sender) client-ip=209.85.220.182; Received: by mail-vc0-f182.google.com with SMTP id hy4so1906019vcb.27 for ; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) X-Received: by 10.52.37.81 with SMTP id w17mr2189886vdj.95.1405522308690; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.37.5 with SMTP id tc5csp305265vcb; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) X-Received: by 10.52.166.10 with SMTP id zc10mr7210793vdb.61.1405522308215; Wed, 16 Jul 2014 07:51:48 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id gz7si8700008veb.16.2014.07.16.07.51.47 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 16 Jul 2014 07:51:48 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X7QWa-00035x-CX; Wed, 16 Jul 2014 14:49:40 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1X7QWY-00035b-P7 for xen-devel@lists.xen.org; Wed, 16 Jul 2014 14:49:38 +0000 Received: from [85.158.143.35:8469] by server-2.bemta-4.messagelabs.com id B3/16-26128-20196C35; Wed, 16 Jul 2014 14:49:38 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-12.tower-21.messagelabs.com!1405522174!18133031!1 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 8098 invoked from network); 16 Jul 2014 14:49:37 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-12.tower-21.messagelabs.com with RC4-SHA encrypted SMTP; 16 Jul 2014 14:49:37 -0000 X-IronPort-AV: E=Sophos;i="5.01,673,1400025600"; d="scan'208";a="153019682" Received: from accessns.citrite.net (HELO FTLPEX01CL02.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 16 Jul 2014 14:49:03 +0000 Received: from kazak.uk.xensource.com (10.80.2.80) by FTLPEX01CL02.citrite.net (10.13.107.79) with Microsoft SMTP Server id 14.3.181.6; Wed, 16 Jul 2014 10:49:02 -0400 Message-ID: <1405522141.767.0.camel@kazak.uk.xensource.com> From: Ian Campbell To: Anup Patel Date: Wed, 16 Jul 2014 15:49:01 +0100 In-Reply-To: <1405506735-19025-1-git-send-email-anup.patel@linaro.org> References: <1405506735-19025-1-git-send-email-anup.patel@linaro.org> Organization: Citrix Systems, Inc. X-Mailer: Evolution 3.12.2-1 MIME-Version: 1.0 X-Originating-IP: [10.80.2.80] X-DLP: MIA1 Cc: Pranavkumar Sawargaonkar , patches@apm.com, stefano.stabellini@eu.citrix.com, stefano.stabellini@citrix.com, xen-devel@lists.xen.org Subject: Re: [Xen-devel] [PATCH v2] xen/arm: Trap and yield on WFE instructions X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.182 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: On Wed, 2014-07-16 at 16:02 +0530, Anup Patel wrote: > If we have a Guest/DomU with two or more of its VCPUs running > on same host CPU then it can quite likely happen that these > VCPUs fight for same spinlock and one of them will waste CPU > cycles in WFE instruction. This patch makes WFE instruction > trap for VCPU and forces VCPU to yield its timeslice. > > The KVM ARM/ARM64 also does similar thing for handling WFE > instructions. (Please refer, > https://lists.cs.columbia.edu/pipermail/kvmarm/2013-November/006259.html) > > In general, this patch is more of an optimization for an > oversubscribed system having number of VCPUs more than > underlying host CPUs. > > Changes since V1: > - Added separate member in union hsr for decoding WFI/WFE > related info. > > Signed-off-by: Anup Patel > Signed-off-by: Pranavkumar Sawargaonkar > Tested-by: Pranavkumar Sawargaonkar Acked + applied. There was a conflict with "[PATCH v4 1/2] xen/arm : Adding helper function for WFI" which I just applied before it. I fixed it up and the result is below, please check it is ok. I also nuked a hard tab which had snuck in. Ian. commit af82c49116c7bf6857be6bf6b56094b9eb2ef012 Author: Anup Patel Date: Wed Jul 16 16:02:15 2014 +0530 xen/arm: Trap and yield on WFE instructions If we have a Guest/DomU with two or more of its VCPUs running on same host CPU then it can quite likely happen that these VCPUs fight for same spinlock and one of them will waste CPU cycles in WFE instruction. This patch makes WFE instruction trap for VCPU and forces VCPU to yield its timeslice. The KVM ARM/ARM64 also does similar thing for handling WFE instructions. (Please refer, https://lists.cs.columbia.edu/pipermail/kvmarm/2013-November/006259.html) In general, this patch is more of an optimization for an oversubscribed system having number of VCPUs more than underlying host CPUs. Signed-off-by: Anup Patel Signed-off-by: Pranavkumar Sawargaonkar Tested-by: Pranavkumar Sawargaonkar Acked-by: Ian Campbell [ ijc -- resolved conflict with "Adding helper function for WFI", nuked stray hard tab ] diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index 5e4c837..3dfabd0 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -90,7 +90,7 @@ void __cpuinit init_traps(void) /* Setup hypervisor traps */ WRITE_SYSREG(HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM| - HCR_TWI|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP, HCR_EL2); + HCR_TWE|HCR_TWI|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP, HCR_EL2); isb(); } @@ -1803,8 +1803,13 @@ asmlinkage void do_trap_hypervisor(struct cpu_user_regs *regs) advance_pc(regs, hsr); return; } - /* at the moment we only trap WFI */ - vcpu_block_unless_event_pending(current); + if ( hsr.wfi_wfe.ti ) { + /* Yield the VCPU for WFE */ + vcpu_force_reschedule(current); + } else { + /* Block the VCPU for WFI */ + vcpu_block_unless_event_pending(current); + } advance_pc(regs, hsr); break; case HSR_EC_CP15_32: diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h index bdfff4e..9d230f3 100644 --- a/xen/include/asm-arm/processor.h +++ b/xen/include/asm-arm/processor.h @@ -276,6 +276,15 @@ union hsr { unsigned long ec:6; /* Exception Class */ } cond; + struct hsr_wfi_wfe { + unsigned long ti:1; /* Trapped instruction */ + unsigned long sbzp:19; + unsigned long cc:4; /* Condition Code */ + unsigned long ccvalid:1;/* CC Valid */ + unsigned long len:1; /* Instruction length */ + unsigned long ec:6; /* Exception Class */ + } wfi_wfe; + /* reg, reg0, reg1 are 4 bits on AArch32, the fifth bit is sbzp. */ struct hsr_cp32 { unsigned long read:1; /* Direction */