From patchwork Fri Feb 28 11:52:54 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrew Jones X-Patchwork-Id: 25531 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qc0-f200.google.com (mail-qc0-f200.google.com [209.85.216.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id C05FA20543 for ; Fri, 28 Feb 2014 11:53:48 +0000 (UTC) Received: by mail-qc0-f200.google.com with SMTP id c9sf1294555qcz.11 for ; Fri, 28 Feb 2014 03:53:48 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=lAdpS3vyQx6JxRd+anWNScWWSwQj80bnIgXw7+9xTeE=; b=RsYXmJtkdZN5GF+yS71WvygII1UqtRY7bmccsWYBNeDWCuGju1Ro2qkM/EInLhSe/b cniY+KPA4SopEkgmAC8MUO0eIe3/bXbaCINjf/qrj7hHjr9bv7fE9ntepJLVG+5pZFW7 zz7dGnHRjcdhRDrmPuMal0eTQGXrV5gztH0imQ4CmDpeUHbKZ6JGKILClUZiwbsBIeeE QK/OTQYVf9s84C7CQXlsLYQ91Fvv7dBVe2AWjRUoAaBw4Bn7hxYJCbHDd38oBUrDkzpu YQiPuZ1SfqWMvCEXnkChrxa3EMeJd5Y49H/R7p2K+gewt7ML+WXGkvtB//3ei65fuHHJ jhjg== X-Gm-Message-State: ALoCoQnCzjm06tBJZ6ybbTnvRP7JyXVIZtz6KjdtmDRWDuRW/JbjAJmXlbff2WlarF5G2wp+mY5p X-Received: by 10.236.90.116 with SMTP id d80mr795120yhf.16.1393588428563; Fri, 28 Feb 2014 03:53:48 -0800 (PST) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.25.33 with SMTP id 30ls1077701qgs.22.gmail; Fri, 28 Feb 2014 03:53:48 -0800 (PST) X-Received: by 10.221.37.1 with SMTP id tc1mr2175711vcb.32.1393588428329; Fri, 28 Feb 2014 03:53:48 -0800 (PST) Received: from mail-vc0-f179.google.com (mail-vc0-f179.google.com [209.85.220.179]) by mx.google.com with ESMTPS id u5si186866vdo.43.2014.02.28.03.53.48 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 28 Feb 2014 03:53:48 -0800 (PST) Received-SPF: neutral (google.com: 209.85.220.179 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.179; Received: by mail-vc0-f179.google.com with SMTP id lh14so591300vcb.24 for ; Fri, 28 Feb 2014 03:53:48 -0800 (PST) X-Received: by 10.220.103.141 with SMTP id k13mr2122829vco.25.1393588428230; Fri, 28 Feb 2014 03:53:48 -0800 (PST) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.130.193 with SMTP id u1csp48187vcs; Fri, 28 Feb 2014 03:53:47 -0800 (PST) X-Received: by 10.66.192.162 with SMTP id hh2mr2975762pac.150.1393588427260; Fri, 28 Feb 2014 03:53:47 -0800 (PST) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i7si1794433pav.101.2014.02.28.03.53.46; Fri, 28 Feb 2014 03:53:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752302AbaB1Lxi (ORCPT + 26 others); Fri, 28 Feb 2014 06:53:38 -0500 Received: from mx1.redhat.com ([209.132.183.28]:10535 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752139AbaB1LxK (ORCPT ); Fri, 28 Feb 2014 06:53:10 -0500 Received: from int-mx12.intmail.prod.int.phx2.redhat.com (int-mx12.intmail.prod.int.phx2.redhat.com [10.5.11.25]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s1SBr9G1027873 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 28 Feb 2014 06:53:10 -0500 Received: from hawk.usersys.redhat.com.com (dhcp-1-243.brq.redhat.com [10.34.1.243]) by int-mx12.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id s1SBr6Qm029274; Fri, 28 Feb 2014 06:53:08 -0500 From: Andrew Jones To: kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mtosatti@redhat.com, pbonzini@redhat.com Subject: [PATCH 1/2 v2] x86: kvm: rate-limit global clock updates Date: Fri, 28 Feb 2014 12:52:54 +0100 Message-Id: <1393588375-13664-2-git-send-email-drjones@redhat.com> In-Reply-To: <1393588375-13664-1-git-send-email-drjones@redhat.com> References: <1393588375-13664-1-git-send-email-drjones@redhat.com> X-Scanned-By: MIMEDefang 2.68 on 10.5.11.25 Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: drjones@redhat.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.179 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , When we update a vcpu's local clock it may pick up an NTP correction. We can't wait an indeterminate amount of time for other vcpus to pick up that correction, so commit 0061d53daf26f introduced a global clock update. However, we can't request a global clock update on every vcpu load either (which is what happens if the tsc is marked as unstable). The solution is to rate-limit the global clock updates. Marcelo calculated that we should delay the global clock updates no more than 0.1s as follows: Assume an NTP correction c is applied to one vcpu, but not the other, then in n seconds the delta of the vcpu system_timestamps will be c * n. If we assume a correction of 500ppm (worst-case), then the two vcpus will diverge 50us in 0.1s, which is a considerable amount. Signed-off-by: Andrew Jones --- v2: switch from kvm_{get,put}_kvm() to cancel_delayed_work_sync() arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/x86.c | 27 +++++++++++++++++++++++---- 2 files changed, 24 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index e714f8c08ccf2..9aa09d330a4b5 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -598,6 +598,7 @@ struct kvm_arch { bool use_master_clock; u64 master_kernel_ns; cycle_t master_cycle_now; + struct delayed_work kvmclock_update_work; struct kvm_xen_hvm_config xen_hvm_config; diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 4cca45853dfeb..e656719750fdf 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -1628,14 +1628,21 @@ static int kvm_guest_time_update(struct kvm_vcpu *v) * the others. * * So in those cases, request a kvmclock update for all vcpus. - * The worst case for a remote vcpu to update its kvmclock - * is then bounded by maximum nohz sleep latency. + * We need to rate-limit these requests though, as they can + * considerably slow guests that have a large number of vcpus. + * The time for a remote vcpu to update its kvmclock is bound + * by the delay we use to rate-limit the updates. */ -static void kvm_gen_kvmclock_update(struct kvm_vcpu *v) +#define KVMCLOCK_UPDATE_DELAY msecs_to_jiffies(100) + +static void kvmclock_update_fn(struct work_struct *work) { int i; - struct kvm *kvm = v->kvm; + struct delayed_work *dwork = to_delayed_work(work); + struct kvm_arch *ka = container_of(dwork, struct kvm_arch, + kvmclock_update_work); + struct kvm *kvm = container_of(ka, struct kvm, arch); struct kvm_vcpu *vcpu; kvm_for_each_vcpu(i, vcpu, kvm) { @@ -1644,6 +1651,15 @@ static void kvm_gen_kvmclock_update(struct kvm_vcpu *v) } } +static void kvm_gen_kvmclock_update(struct kvm_vcpu *v) +{ + struct kvm *kvm = v->kvm; + + set_bit(KVM_REQ_CLOCK_UPDATE, &v->requests); + schedule_delayed_work(&kvm->arch.kvmclock_update_work, + KVMCLOCK_UPDATE_DELAY); +} + static bool msr_mtrr_valid(unsigned msr) { switch (msr) { @@ -7019,6 +7035,8 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type) pvclock_update_vm_gtod_copy(kvm); + INIT_DELAYED_WORK(&kvm->arch.kvmclock_update_work, kvmclock_update_fn); + return 0; } @@ -7056,6 +7074,7 @@ static void kvm_free_vcpus(struct kvm *kvm) void kvm_arch_sync_events(struct kvm *kvm) { + cancel_delayed_work_sync(&kvm->arch.kvmclock_update_work); kvm_free_all_assigned_devices(kvm); kvm_free_pit(kvm); }