From patchwork Thu Jun 8 23:44:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 103434 Delivered-To: patches@linaro.org Received: by 10.140.91.77 with SMTP id y71csp2677786qgd; Thu, 8 Jun 2017 16:51:07 -0700 (PDT) X-Received: by 10.98.196.86 with SMTP id y83mr34842302pff.97.1496965867884; Thu, 08 Jun 2017 16:51:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1496965867; cv=none; d=google.com; s=arc-20160816; b=krefQ8ZFH6YJBBWz9FHUtV2IxYnxpTvxHSRDha+pfUIjf5Zf1Zn/DU3KjAlkRjYlMb 6jRuAoLizPVaxR6SixFdt7RL0KYQ/wYwvFqi5xx8gCQOiPeQuNAy8DJ67IjbAbCmnKS8 ScszIJyh/u5nHboXJes3vVRa0fDaVJo/HYBO2crqUf6fWBc23N2zbALiiBQ6qZY+FQk4 ez8O0onYPh2JnCM4VgKf0H4Vm8f0+pty4ggUnGWkMgSJPVbKKPS/UeAK796H5MUIcvoN 8ppmIcsz3BiRQLktyL8HJwHZqbbFtewkeqpv06zs1dGwdHgO0CGvJpx1ra9APHIhEp8s Dfxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=RyFOfvvZr6OIGvU2ycIzY0udj5cUmEPilU1x1Zeo2+U=; b=uDYuYqThNplcK9wm+zxNAZ0jhJYqd1gOccOj7oiCpsA+h63AxV4w/WqSHnm4hM/rJg G85BtycfxJlhDC7xlO4I6pXr14QOnTEmRgUEY+lkxMY7gKRHtd5JM6uYItY0IpqzPEN9 hZ3Y6OR7PHhtWaSQcXVVbTzdG5zlPFQOwFIPTKd2hoesD/wKJj7L94zTj3zBpWGzuhPH RYtPEPIqOyk53aI8zwJb2sPwyTwfpSTrLmIIqGY45JPafY15utWhlupSIwuAyMTitjml JeNihysjGdvL4FjOea7mzBa+rMqMe7R4R/pDDufgBDys9HVFOE6LBWzPBO4/oapzToyn U3zQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.41 as permitted sender) smtp.mailfrom=john.stultz@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41]) by mx.google.com with SMTPS id i128sor3941570pgd.127.2017.06.08.16.51.07 for (Google Transport Security); Thu, 08 Jun 2017 16:51:07 -0700 (PDT) Received-SPF: pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: domain of john.stultz@linaro.org designates 209.85.220.41 as permitted sender) smtp.mailfrom=john.stultz@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=RyFOfvvZr6OIGvU2ycIzY0udj5cUmEPilU1x1Zeo2+U=; b=NS7hXj7v8FkpHs3eaXDUHvELTDz5Ma3mWP3s7vC6VknOJlXE4FYDHAdni6UzqqanAD 6Ls7fGb0ucbuCVZCGxb/T6LZSawgGq4QdvBmouDtIfXeyGfQdGYmS6NjAen4R7wvb4/X FemDDjmzIZ8Xw5CkkIGHn5nbi1MXu4Tp4NnZs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=RyFOfvvZr6OIGvU2ycIzY0udj5cUmEPilU1x1Zeo2+U=; b=f8thuhnYgIrHMA6DHIaPuHCjSEZXD5LklNrn12iczR2EhnBJ9iVb8ISaD8CvbpHnAm 7X7uF2Cb0KSzkwhy07HOkqnkydxUEniPpCAoZ8ywXdoc49QHoWKqHKg262oPmFst2Xnd HL68NdUpqETENjT9F0ieopw2eKVDoKO/9fhFLP6oyuvA5VPZRUmnHghejVpHqGemCMgx 7uuRR4pEDN0IbE+Osz+zsQJokz+nHJ1wDnsL6HQssBZB7wUr9hq0tBWMGqFDV1x6fa29 7fuhmBTVZULvfIUmuJTdedIVj9RQnux5LxRiA1U/GyoeubGCUIvdTwCdtusQGJ0ZBohX 4fTQ== X-Gm-Message-State: AODbwcCrj8kUpD+IMqaKEDtEO3FNSlcP8ML2KiAwmxgrl9IPMb5HocSJ 78cUpm4uEBvuU4tcIiQ= X-Received: by 10.101.76.11 with SMTP id u11mr40197573pgq.109.1496965867594; Thu, 08 Jun 2017 16:51:07 -0700 (PDT) Return-Path: Received: from localhost.localdomain ([2601:1c2:1002:83f0:4e72:b9ff:fe99:466a]) by smtp.gmail.com with ESMTPSA id b86sm14462850pfc.27.2017.06.08.16.51.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 08 Jun 2017 16:51:06 -0700 (PDT) From: John Stultz To: lkml Cc: John Stultz , Thomas Gleixner , Ingo Molnar , Miroslav Lichvar , Richard Cochran , Prarit Bhargava , Stephen Boyd , Kevin Brodsky , Will Deacon , Daniel Mentz , "stable #4 . 8+" Subject: [PATCH 2/3 v3] time: Fix CLOCK_MONOTONIC_RAW sub-nanosecond accounting Date: Thu, 8 Jun 2017 16:44:21 -0700 Message-Id: <1496965462-20003-3-git-send-email-john.stultz@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1496965462-20003-1-git-send-email-john.stultz@linaro.org> References: <1496965462-20003-1-git-send-email-john.stultz@linaro.org> Due to how the MONOTONIC_RAW accumulation logic was handled, there is the potential for a 1ns discontinuity when we do accumulations. This small discontinuity has for the most part gone un-noticed, but since ARM64 enabled CLOCK_MONOTONIC_RAW in their vDSO clock_gettime implementation, we've seen failures with the inconsistency-check test in kselftest. This patch addresses the issue by using the same sub-ns accumulation handling that CLOCK_MONOTONIC uses, which avoids the issue for in-kernel users. Since the ARM64 vDSO implementation has its own clock_gettime calculation logic, this patch reduces the frequency of errors, but failures are still seen. The ARM64 vDSO will need to be updated to include the sub-nanosecond xtime_nsec values in its calculation for this issue to be completely fixed. Cc: Thomas Gleixner Cc: Ingo Molnar Cc: Miroslav Lichvar Cc: Richard Cochran Cc: Prarit Bhargava Cc: Stephen Boyd Cc: Kevin Brodsky Cc: Will Deacon Cc: Daniel Mentz Cc: stable #4.8+ Tested-by: Daniel Mentz Signed-off-by: John Stultz --- v2: Address Ingo's style feedback --- include/linux/timekeeper_internal.h | 4 ++-- kernel/time/timekeeping.c | 19 ++++++++++--------- 2 files changed, 12 insertions(+), 11 deletions(-) -- 2.7.4 diff --git a/include/linux/timekeeper_internal.h b/include/linux/timekeeper_internal.h index e9834ad..f7043cc 100644 --- a/include/linux/timekeeper_internal.h +++ b/include/linux/timekeeper_internal.h @@ -57,7 +57,7 @@ struct tk_read_base { * interval. * @xtime_remainder: Shifted nano seconds left over when rounding * @cycle_interval - * @raw_interval: Raw nano seconds accumulated per NTP interval. + * @raw_interval: Shifted raw nano seconds accumulated per NTP interval. * @ntp_error: Difference between accumulated time and NTP time in ntp * shifted nano seconds. * @ntp_error_shift: Shift conversion between clock shifted nano seconds and @@ -99,7 +99,7 @@ struct timekeeper { u64 cycle_interval; u64 xtime_interval; s64 xtime_remainder; - u32 raw_interval; + u64 raw_interval; /* The ntp_tick_length() value currently being used. * This cached copy ensures we consistently apply the tick * length for an entire tick, as ntp_tick_length may change diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index eff94cb..b602c48 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -280,7 +280,7 @@ static void tk_setup_internals(struct timekeeper *tk, struct clocksource *clock) /* Go back from cycles -> shifted ns */ tk->xtime_interval = interval * clock->mult; tk->xtime_remainder = ntpinterval - tk->xtime_interval; - tk->raw_interval = (interval * clock->mult) >> clock->shift; + tk->raw_interval = interval * clock->mult; /* if changing clocks, convert xtime_nsec shift units */ if (old_clock) { @@ -1996,7 +1996,7 @@ static u64 logarithmic_accumulation(struct timekeeper *tk, u64 offset, u32 shift, unsigned int *clock_set) { u64 interval = tk->cycle_interval << shift; - u64 raw_nsecs; + u64 snsec_per_sec; /* If the offset is smaller than a shifted interval, do nothing */ if (offset < interval) @@ -2011,14 +2011,15 @@ static u64 logarithmic_accumulation(struct timekeeper *tk, u64 offset, *clock_set |= accumulate_nsecs_to_secs(tk); /* Accumulate raw time */ - raw_nsecs = (u64)tk->raw_interval << shift; - raw_nsecs += tk->raw_time.tv_nsec; - if (raw_nsecs >= NSEC_PER_SEC) { - u64 raw_secs = raw_nsecs; - raw_nsecs = do_div(raw_secs, NSEC_PER_SEC); - tk->raw_time.tv_sec += raw_secs; + tk->tkr_raw.xtime_nsec += (u64)tk->raw_time.tv_nsec << tk->tkr_raw.shift; + tk->tkr_raw.xtime_nsec += tk->raw_interval << shift; + snsec_per_sec = (u64)NSEC_PER_SEC << tk->tkr_raw.shift; + while (tk->tkr_raw.xtime_nsec >= snsec_per_sec) { + tk->tkr_raw.xtime_nsec -= snsec_per_sec; + tk->raw_time.tv_sec++; } - tk->raw_time.tv_nsec = raw_nsecs; + tk->raw_time.tv_nsec = tk->tkr_raw.xtime_nsec >> tk->tkr_raw.shift; + tk->tkr_raw.xtime_nsec -= (u64)tk->raw_time.tv_nsec << tk->tkr_raw.shift; /* Accumulate error between NTP and clock interval */ tk->ntp_error += tk->ntp_tick << shift;