From patchwork Fri May 5 10:38:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98609 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp49648qge; Fri, 5 May 2017 03:39:11 -0700 (PDT) X-Received: by 10.55.54.202 with SMTP id d193mr13097702qka.160.1493980751823; Fri, 05 May 2017 03:39:11 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id o63si4172303qka.230.2017.05.05.03.39.11 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:39:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46169 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6adh-0002YG-5e for patch@linaro.org; Fri, 05 May 2017 06:39:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52548) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acO-0001kL-Os for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acL-0006hL-IV for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:48 -0400 Received: from mail-wr0-x22a.google.com ([2a00:1450:400c:c0c::22a]:36447) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acL-0006gy-BI for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:45 -0400 Received: by mail-wr0-x22a.google.com with SMTP id l50so1360655wrc.3 for ; Fri, 05 May 2017 03:37:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=XQLHoQCfjYprlYkp8C9uZ/tNq5f1xqPmuCGp4WYUzw4=; b=SVoM7i9U9HlEGyk3LYnPNN0jZnXcFRR05FTzO9G3lsQV9xoK7/TQKv0tflBcQTMoTl AuyR3bO05wLgBEuoiTuudfdeZbEz9UJAVBdzj9WK2lZxTp3hc4aeH+JxJAe10HxTj3LJ 5Lvu7Uqf3OSczqL4u61KVoBxHomWnoNKf52XY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=XQLHoQCfjYprlYkp8C9uZ/tNq5f1xqPmuCGp4WYUzw4=; b=pPkWrBdfrBOm536SaqWxU8cC0mwRizrkZyILFAAKhmNJMo7grEJdiqSpT/NwJ/c5sY JzVCK3RZa8uH1Yi2PPIYlT0gV9O9ZBlssQ0OBy0RI5QDuFz2TrXyUI7MQKYVKrlOS1je OFSqgCAgZHPv/U1dxZvBkwEeUMYeDOYK5qVXWd1EE83XlqgzMZPGkYZSTwRZlKu2PD+t PQQOhvRUVztaeZZR2H7EoW+RWDkDNrT5OdfGwPsAxxrB9/6qlUWyg3pK7Sq/mAnYDUKW LNfEl4eErY2Yp07XQ28ong71LCmMl1wPwpTMBzn7kqMMtFcNGA2gXMX9f7/YeKbrKhXE u2WQ== X-Gm-Message-State: AN3rC/7L6WdFoEDue8zkCOCmYGsF3Lnrq+CMWyXs0PvyT5/u5WZUnfd2 a05rq21ed+lNUHEx X-Received: by 10.223.134.80 with SMTP id 16mr22507370wrw.62.1493980662605; Fri, 05 May 2017 03:37:42 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id y3sm3989156wrd.23.2017.05.05.03.37.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:41 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id A9DD83E0192; Fri, 5 May 2017 11:38:22 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:13 +0100 Message-Id: <20170505103822.20641-1-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::22a Subject: [Qemu-devel] [RFC PATCH v1 0/9] BQL and Replay Lock changes X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Hi, This RFC does two principle things. It continues the push to reduce the BQL lock contention to a minimum by dropping the BQL for most of the run loop - eventually only holding it for timer processing and sleeping. The second part is an attempt to fix the breakage caused to record/replay by the previous work reducing the BQL in 2.9. The patch breakdown is as follows: - target/arm/arm-powertctl: drop BQL assertions This just fixes a bogus assert. The async work doesn't need BQL protection as it is all done in the context of the vCPU (meaning nothing else should be messing with it). However going forward we will need to audit all the async work calls to make sure they are fine to run outside of the BQL. - cpus: push BQL lock to qemu_*_wait_io_event - cpus: only take BQL for sleeping threads The BQL reduction is done in two stages, mainly to make bisection easier. - replay/replay-internal.c: track holding of replay_lock - replay: make locking visible outside replay code - replay: push replay_mutex_lock up the call tree There is still more work to do here but essentially replay_lock now replaces the BQL in keeping synchronisation between main-loop and the TCG thread - it is no longer a fine lock for the replay log but a gross lock that keeps batches of updates together. So far I have been testing with the simple testcase: ./arm-softmmu/qemu-system-arm -machine type=vexpress-a9 -m 1024 \ -display none -smp 1 -kernel ../images/vexpress-kernel.img \ -dtb ../images/vexpress-v2p-ca9.dtb \ -append "console=ttyAMA0" -serial mon:stdio \ -icount shift=7,rr=record,rrfile=replay.bin ..and that works fine. However more complex testing with more devices to exercise the async events code is needed here. One change that needs some careful care is we are now not dropping the replay_mutex lock between events. And finally: - scripts/qemu-gdb: add simple tcg lock status helper - util/qemu-thread-*: add qemu_lock, locked and unlock trace events - scripts/analyse-locks-simpletrace.py: script to analyse lock times These are helpers I wrote while debugging the locking. The GDB helper is fairly dump but does attempt to show where locks are held. The qemu-lock tracing is a little hacky but could prove useful in doing more detailed lock analysis. Any thoughts/comments? Does this seem like a reasonable direction to go? Alex Bennée (9): target/arm/arm-powertctl: drop BQL assertions cpus: push BQL lock to qemu_*_wait_io_event cpus: only take BQL for sleeping threads replay/replay-internal.c: track holding of replay_lock replay: make locking visible outside replay code replay: push replay_mutex_lock up the call tree scripts/qemu-gdb: add simple tcg lock status helper util/qemu-thread-*: add qemu_lock, locked and unlock trace events scripts/analyse-locks-simpletrace.py: script to analyse lock times cpus-common.c | 13 ++--- cpus.c | 57 ++++++++++++++++++--- docs/replay.txt | 19 +++++++ include/qemu/thread.h | 7 ++- include/sysemu/replay.h | 16 ++++++ kvm-all.c | 4 -- replay/replay-char.c | 21 +++----- replay/replay-events.c | 18 ++----- replay/replay-internal.c | 22 +++++++- replay/replay-internal.h | 5 +- replay/replay-time.c | 10 ++-- replay/replay.c | 40 +++++++-------- scripts/analyse-locks-simpletrace.py | 99 ++++++++++++++++++++++++++++++++++++ scripts/qemu-gdb.py | 3 +- scripts/qemugdb/tcg.py | 46 +++++++++++++++++ stubs/replay.c | 15 ++++++ target/arm/arm-powerctl.c | 8 --- target/i386/hax-all.c | 2 - util/main-loop.c | 23 +++++++-- util/qemu-thread-posix.c | 11 +++- util/trace-events | 5 ++ vl.c | 2 + 22 files changed, 353 insertions(+), 93 deletions(-) create mode 100755 scripts/analyse-locks-simpletrace.py create mode 100644 scripts/qemugdb/tcg.py -- 2.11.0