From patchwork Wed Sep 4 14:49:56 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Newton X-Patchwork-Id: 19756 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f200.google.com (mail-lb0-f200.google.com [209.85.217.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 1EFB425AC6 for ; Wed, 4 Sep 2013 14:50:01 +0000 (UTC) Received: by mail-lb0-f200.google.com with SMTP id y6sf436096lbh.7 for ; Wed, 04 Sep 2013 07:50:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:message-id:date:from:user-agent :mime-version:to:cc:subject:x-original-sender :x-original-authentication-results:precedence:mailing-list:list-id :list-post:list-help:list-archive:list-unsubscribe:content-type :content-transfer-encoding; bh=ECnMQK1p/k2u1z5+3uFkTus12qfHICh00k+jPs7FZ90=; b=ERTASvS2mFJzWtF2hVC6uFF+oDYlaVOb1S0wwT4QllzgYcrLggfomMY3iK2ozXNko/ NtN4JVtnuP1dUqKxHsXV6lKNGgO+kvJ4YO6gBr0lNfmg1IiiHOok9Vt+zNIadHOWkvH+ mGAbXFmkJV/6XPrte4uCAbBtF8YIscoup0yf9vtQZ9D2Se1ndn8IVq0BGNQfXkAtFSAR aqOYSEUgIrDqmI9skFQ2aTFfci4LApzKMOZQpQLK0XxqIVVlt1/o2MmS0ytQioZAlAQN /iw9eIut2gNbTxQoAROEprWc9NQuk2dbh0VN0zPOMhOpGCVBtQdpFJitruuHkihAEMOq dNIw== X-Received: by 10.112.50.3 with SMTP id y3mr596179lbn.5.1378306199955; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.9.99 with SMTP id y3ls17174laa.60.gmail; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) X-Received: by 10.112.156.166 with SMTP id wf6mr2780741lbb.13.1378306199774; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) Received: from mail-la0-f48.google.com (mail-la0-f48.google.com [209.85.215.48]) by mx.google.com with ESMTPS id a9si10840882lbv.134.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 04 Sep 2013 07:49:59 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.215.48 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.215.48; Received: by mail-la0-f48.google.com with SMTP id er20so390514lab.7 for ; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) X-Gm-Message-State: ALoCoQn/4Hu0uXD6TduQ8wbRCmunioMH0bphgKR7P4uiGh8DIsAImfNlLSAeGqAkOiB9ErLAg9Sn X-Received: by 10.112.14.102 with SMTP id o6mr2756553lbc.28.1378306199584; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patches@linaro.org Received: by 10.112.11.201 with SMTP id s9csp229495lbb; Wed, 4 Sep 2013 07:49:59 -0700 (PDT) X-Received: by 10.112.60.104 with SMTP id g8mr2368555lbr.32.1378306199155; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) Received: from mail-la0-f53.google.com (mail-la0-f53.google.com [209.85.215.53]) by mx.google.com with ESMTPS id l7si10844445lbd.110.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 04 Sep 2013 07:49:59 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.215.53 is neither permitted nor denied by best guess record for domain of will.newton@linaro.org) client-ip=209.85.215.53; Received: by mail-la0-f53.google.com with SMTP id el20so405361lab.12 for ; Wed, 04 Sep 2013 07:49:59 -0700 (PDT) X-Received: by 10.152.120.228 with SMTP id lf4mr1612396lab.44.1378306198937; Wed, 04 Sep 2013 07:49:58 -0700 (PDT) Received: from localhost.localdomain (cpc6-seac21-2-0-cust453.7-2.cable.virginmedia.com. [82.1.113.198]) by mx.google.com with ESMTPSA id k6sm11795927lae.9.1969.12.31.16.00.00 (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 04 Sep 2013 07:49:57 -0700 (PDT) Message-ID: <52274894.4000604@linaro.org> Date: Wed, 04 Sep 2013 15:49:56 +0100 From: Will Newton User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130805 Thunderbird/17.0.8 MIME-Version: 1.0 To: libc-alpha@sourceware.org CC: patches@linaro.org Subject: [RFC PATCH] benchtests/bench-plot.py: Add graphing script for string benchmarks. X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: will.newton@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.215.48 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Precedence: list Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org List-ID: X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Add a Python script that uses pylab to graph the results of string benchmarks. Currently it only supports graphing the results of the strlen and memcpy benchmarks. ChangeLog: 2013-09-03 Will Newton * benchtests/bench-plot.py: New file. --- benchtests/bench-plot.py | 184 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 184 insertions(+) create mode 100755 benchtests/bench-plot.py diff --git a/benchtests/bench-plot.py b/benchtests/bench-plot.py new file mode 100755 index 0000000..7fd680b --- /dev/null +++ b/benchtests/bench-plot.py @@ -0,0 +1,184 @@ +#!/usr/bin/env python +# Copyright (C) 2013 Free Software Foundation, Inc. +# This file is part of the GNU C Library. + +# The GNU C Library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. + +# The GNU C Library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. + +# You should have received a copy of the GNU Lesser General Public +# License along with the GNU C Library; if not, see +# . + +# A script for graphing the results of glibc string benchmarks. + +import glob +import math +import os +import re +import sys + +import pylab + +def unique_values(rows, column): + values = [] + for row in rows: + if not row[column] in values: + values.append(row[column]) + return sorted(values) + +def make_colours(): + return iter('m b g r c y k pink orange brown grey'.split()) + +def pretty_kb(v): + if v < 1024: + return '%d' % v + else: + if v % 1024 == 0: + return '%d k' % (v//1024) + else: + return '%.1f k' % (v/1024) + +def plot_null(benchmark, input_files, oldinput_files, output_path): + pass + +def parse_benchmark_rows(input_files): + impls = [] + rows = [] + matcher = re.compile("Length\s+(\d+), alignment\s+([0-9 /]+):") + for input_file in input_files: + lines = open(input_file).readlines() + if not impls: + impls = lines[0].strip().split("\t") + for line in lines[1:]: + columns = line.split("\t") + m = matcher.match(columns[0]) + groups = m.groups() + length = groups[0] + alignment = groups[1] + rows.append([int(length), alignment] + map(float, columns[1:])) + return (impls, rows) + +def plot_benchmark(benchmark, input_files, oldinput_files, output_path): + (newimpls, newrows) = parse_benchmark_rows(input_files) + (oldimpls, oldrows) = parse_benchmark_rows(oldinput_files) + # We're only interested in the glibc impl for the old files + if oldimpls: + oldimpls = ["%s (old)" % oldimpls[0]] + alignments = unique_values(newrows, 1) + for alignment in alignments: + # Draw one figure per alignment value + pylab.figure(1).set_size_inches((12, 10)) + pylab.clf() + plot_done = False + colours = make_colours() + for (impls, rows) in ((newimpls, newrows), (oldimpls, oldrows)): + if not rows: + continue + plot_rows = [] + for row in rows: + if row[1] == alignment: + plot_rows.append(row) + X = unique_values(plot_rows, 0) + # Filter out zero length entries + X = [x for x in X if x > 0] + # If there are too few data points, skip this alignment + if len(X) < 2: + continue + numimpls = len(impls) + Y = [] + Yerr = [] + # Initialize the Y and Yerr arrays + for i in range(0, numimpls): + Y.append([]) + Yerr.append([[], []]) + for length in X: + matches = [x for x in plot_rows if x[0] == length] + # If there is more than one test run for a given test + # then calculate the mean and min/max + if len(matches) > 1: + for i in range(0, numimpls): + vals = [x[2 + i] for x in matches] + mean = length / (sum(vals)/len(vals)) + Y[i].append(mean) + err1 = (length / max(vals)) - mean + if err1 < 0: + err1 = 0 + err2 = (length / min(vals)) - mean + if err2 > 0: + err2 = 0 + Yerr[i][0].append(abs(err2)) + Yerr[i][1].append(err1) + else: + for i in range(0, numimpls): + Y[i].append(float(length) / matches[2 + i]) + i = 0 + for impl in impls: + colour = colours.next() + pylab.plot(X, Y[i], c=colour) + plot_done = True + # If we have error values then draw the bars + if len(Yerr[i]) > 0: + pylab.errorbar(X, Y[i], yerr=Yerr[i], c=colour, label=impl, + fmt='o') + else: + pylab.scatter(X, Y[i], c=colour, label=impl, + edgecolors='none') + i += 1 + # If we didn't draw anything then skip this graph + if not plot_done: + continue + pylab.legend(loc='upper left', ncol=3, prop={'size': 'small'}) + pylab.grid() + # Tidy up alignment text for benchmarks that need it + alignment = alignment.replace(" ", "") + alignment = alignment.replace("/", "-") + pylab.title('%s of %s byte aligned buffers' % (benchmark, alignment)) + pylab.xlabel('Length (B)') + pylab.ylabel('Bytes per cycle or ns') + + top = max(X) + + power = int(round(math.log(top) / math.log(2))) + + pylab.semilogx() + + pylab.axes().set_xticks([2**x for x in range(0, power+1)]) + pylab.axes().set_xticklabels([pretty_kb(2**x) + for x in range(0, power+1)]) + pylab.xlim(0, top) + pylab.ylim(0, pylab.ylim()[1]) + pylab.savefig(os.path.join(output_path, '%s-%s.png' % + (benchmark, alignment)), dpi=72) + +plotters = { + "memcpy" : plot_benchmark, + "strlen" : plot_benchmark, +} + +def plot_benchmark(directory, benchmark): + plotter = plotters.get(benchmark, plot_null) + input_path = os.path.join(directory, "bench-%s.*.out" % benchmark) + input_files = glob.glob(input_path) + if not input_files: + return + oldinput_path = os.path.join(directory, "bench-%s.*.out.old" % benchmark) + oldinput_files = glob.glob(oldinput_path) + plotter(benchmark, input_files, oldinput_files, directory) + +def usage(): + print "bench-plot.py " + sys.exit(1) + +if __name__ == '__main__': + if len(sys.argv) != 3: + usage() + directory = sys.argv[1] + benchmark = sys.argv[2] + plot_benchmark(directory, benchmark)