From patchwork Wed May 3 14:51:38 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yury Norov X-Patchwork-Id: 98492 Delivered-To: patch@linaro.org Received: by 10.140.89.200 with SMTP id v66csp94692qgd; Wed, 3 May 2017 07:52:22 -0700 (PDT) X-Received: by 10.84.216.26 with SMTP id m26mr31791558pli.79.1493823142911; Wed, 03 May 2017 07:52:22 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y15si2742202pff.9.2017.05.03.07.52.22; Wed, 03 May 2017 07:52:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@CAVIUMNETWORKS.onmicrosoft.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752386AbdECOwK (ORCPT + 25 others); Wed, 3 May 2017 10:52:10 -0400 Received: from mail-bn3nam01on0088.outbound.protection.outlook.com ([104.47.33.88]:23131 "EHLO NAM01-BN3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751118AbdECOwC (ORCPT ); Wed, 3 May 2017 10:52:02 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=8R9YR0X8izFXRb4/mzatujs/Ni2m5cPyMdXdXmrnMuo=; b=eLXOOZvu0Srvd84Q5VDbkxItCRirZQ0xnhvnXz2548wzL6V+WEXN65UJAyFi2FkaV/I5jF36Pt+xLuNBsEsstihz4+Mhbm65TM3h3LNzXlpDOdchoRpsLI4ilN6v93HIrFrtRPZVYUw0qAy1q4+iSyQgN1mHh96x2dlz0bECg9Q= Authentication-Results: arm.com; dkim=none (message not signed) header.d=none; arm.com; dmarc=none action=none header.from=caviumnetworks.com; Received: from localhost (176.59.53.57) by BN3PR0701MB1268.namprd07.prod.outlook.com (10.160.118.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1061.12; Wed, 3 May 2017 14:51:53 +0000 From: Yury Norov To: Will Deacon , Peter Zijlstra , linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Yury Norov , Adam Wallis , Andrew Pinski , Arnd Bergmann , Catalin Marinas , Ingo Molnar , Jan Glauber , Mark Rutland , Pan Xinhui Subject: [PATCH 0/3] arm64: queued spinlocks and rw-locks Date: Wed, 3 May 2017 17:51:38 +0300 Message-Id: <20170503145141.4966-1-ynorov@caviumnetworks.com> X-Mailer: git-send-email 2.11.0 MIME-Version: 1.0 X-Originating-IP: [176.59.53.57] X-ClientProxiedBy: VI1PR0602CA0024.eurprd06.prod.outlook.com (10.175.26.162) To BN3PR0701MB1268.namprd07.prod.outlook.com (10.160.118.142) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f68d4260-adb4-4a41-0394-08d49233f08e X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201703131423075)(201703031133081); SRVR:BN3PR0701MB1268; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1268; 3:yBXP7BPNEEkw8DATm5nm5kJt+jV7eNLkc8q3TiV3JD6638naP9NNQkjSDw3tDKcQXmo3+xBtkOSoBZBHbYhmF474jPBPtQ/khwwslx0UFXXtwLa2/C+EcAJ2G6MSbySIxCQeGX9EXFUkac916SmIqHDDUCfupPrSTo4QepA4vm5wq+oGl0gg4f4SWkkH3Eh1vn4uG/TClmykiMKdfyDD9Fy4dD6XVIA8fFkr/wGzQMvlXm1ivsS9WcjR+Nt7qO2uoeRNDz0/HzpbBaU572EMkpXHM7wvmJzZtkZW9oD9qtzddscJ39JeKIt5FCY+aQZ1ym3jd/gugkB0IbJ1pnVNzg==; 25:/0A5znim1ttXp11z9Z5Bk84UqTfaCOQADhQaVehTYuHcst/w1CxgQU4RP87LxjpTZWvN4ZF70L4b4uVX3mmVHg0HgmtQRBx7N58H8osQ+UzdnjxW/mhhLHnbHASq43rE/35tFNIcXxayYOHArxwxhfYBdoc+0v8hESrA8GHBYH81RdKY/rSivNbUEywISI+ot+WPQ0LtGRTzwWOuTtuuy6vIpF+cImLbLHvDknrQOQOS7W7qxF21N7R3ZMc4gZh2pDEwVe8qO4LE+eal9SBy/CE7GT4nbYE3kCo3lhR09WpTExtDJkPNxmo5/CJ/KvnO9wAx/nqGsjZvrb3CP7QS9Z4qFJ1BPiesh/4+nQOUsd5K0g+rGqm9UllkY5heYOPDAes5Gk2TJUPy5jHxNU4zParCnlxApzSNTMRK4Yu8W1e27E4Hfs/Pa5JfK6l450YB+Oxlg0Av/j3uoPWOpzxz5w== X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1268; 31:XgcKgronaXGD2hnrjmauiU0lM8ak1fvFTFw6m9s7yD3uvAQ9nCgZWPaoh7WRAnhwnkI4wKMC7/cnUmA2dK7V8SAXK6rFRCDEJ7KZclbi4dwBZdzxOMQev+trgLFCfevF3N0J/tD/m2scFCeoSTr3JznisJ4IaxTs5/lugRXsYE/n45OS9VwfjMTt2wUERzxzC9D3FV+jJ9rOYKKp9Z8UBN4y4TxApoMuY6BK5bmw0yU=; 20:Re+gW/ZKort5DsK/xwSQrLKG6FgGYmFJ73Wo6QSK0j4oCXuwr5zRQ/BUxPsW3m12QKMCQYvXD/vZmGeBR7NyK/EleT04qUUESgGF9/j6PifuMC1+4YkxCpQDmKf5c2eKtqUPLM0NNVnHJ3efnms1SeMozN+R0UTnokg9KSquAzMowderukSGDJCDl8Lk69rCaH8Zi2/4FjBzEawCh7THpmAb6gTqEgcXK4LtB7kuq2zaywizkUv+FTpabYZ3yFdbQP9YU5tfxCumzF1WKgBBMWP9i+gTXPHmkS1T6iWKnKjLYMZaHcjKcQFuuJzRu+shTxjtbT7yYVMhJApQetq7zyyqT2t/zMir31HBN6Ks221tPQKua7ZCmOiq0DuJsXMs07RJix5luFEXGP+QOb+p1gqLhTLgIrXglFYYqttz/nuyCJxRW95Gg+VtKh+xHiDZuDTHyDOaSHvoJfsqS4GBdL7EK8r+myJLj+cB1aUT3iyWAp9i4Yqlpbf7l3RYbpMBfCM2ALawELfnaleIH7i/2yVgV8bWTqBs4icTkGldpYK+QEyWnUtxFvmVZw9dThG/32svcsk+4uUBndizX39U5vUpXuavvKDnNnGdsI/rp6g= X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040450)(601004)(2401047)(5005006)(8121501046)(3002001)(10201501046)(93006095)(6041248)(20161123555025)(20161123562025)(20161123564025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123558100)(20161123560025)(6072148); SRVR:BN3PR0701MB1268; BCL:0; PCL:0; RULEID:; SRVR:BN3PR0701MB1268; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1268; 4:H4WxlNTita9I7cmvntLiKOlpOaV9PWsXYvcjVkXqOsleNIMGjsUO4L9ksBmF9F0rIChnXTg/ODIjzn1cZis+42Yaz0BiAbP2txxG94KwS9mZSFkXJ4qu8+yJxE7Pg8aJlplSjMg+633EXoSmvfa+KhWoL/1fVOln1SDB55Gw3PDX6xRZk2RiLk8hP33Ap1MZ+8oQQVuugVu66/3gXLm0BtqLAbuNS/B6oDlKe0HxOiywZpoQWWwrWj2DVvtgnTT8PDxlNpbowxe+H0yBhEelp6mmPoYxwak/8/s9PmhZZoVb5C/hiNbJKHn+qJmDCED+ErRpaRUGzQl/6ip69XZ/1v0YcJiy4C+VtKk5CcpPvMRW9J9U+My1PcBQT56rJCSOcVovpvgONmlGFCGWMVMeEKzusoK6fAnfwYZWJoHx4QMqysGUTzKHVAIWuM5mHwdtO1Qnck642QrxC0o1+mZk0kGisJeuKOGZEhg2x5aPY+dQ91YFjHbCEzfpTV9XIRUJteUL1D2hvGWtLOBMrVATpAxDykI25X9TW7r27vIpNK0/xa+G8NpnjYyZ2/4JT87Q0qRgReGuTjPtkokPl44uB4OJ5M96oxTdgnP1/oOOKN0mtkF3t7ZXcoERVYD7ovFhgLuK+dRunM1vbJiPyOY6pdPvhTG7OafOZo+XDlLt3l1iFjlvJ/fE+jbrdFJkeA95tqxUu2KENwWCpPhAMgyFigBZMHdJSPjTVAH71WS/AbmK4KvQQQzHnngnZSfijt7E1uqnJI5xxQtR4WZ+46JHaA== X-Forefront-PRVS: 029651C7A1 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(4630300001)(6069001)(6009001)(39850400002)(39450400003)(39840400002)(39400400002)(11905935001)(7416002)(5660300001)(25786009)(42882006)(1076002)(6666003)(189998001)(305945005)(478600001)(5003940100001)(4326008)(36756003)(6116002)(50986999)(2906002)(48376002)(7736002)(50466002)(3846002)(6306002)(33646002)(53936002)(47776003)(66066001)(6486002)(38730400002)(8676002)(50226002)(54906002)(6496005)(81166006)(42186005); DIR:OUT; SFP:1101; SCL:1; SRVR:BN3PR0701MB1268; H:localhost; FPR:; SPF:None; MLV:sfv; LANG:en; X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1268; 23:84v6AUQiC/9lB45dl8uws+gmzv4ThzZtl9nOeKfV4ufYtn8j+OWIFxbOtEssMgsCkWWXgzbURLMimjADzLKc72eKrYqJNWdF2rUQCVcvBmkkEhs6PNesBeFN5Wa0F3+GkjxdqYU5sujaZp7759Y39iCkOSGRO8Wad7twxHDG4AW8CtF1tYFZ//jjYt2LtqKjnFM5rjH+fP8XQf7cTSD0hkV1OQKU4YLBwgD0lgmQ2+JYIyd3oeiiLdsfGc1nyoHZNTMz3KTgWaGnAWO4oEYaDQvN2I8bhCTb/ubGXMWXFttSU58G1NPjLVa5OrNviRG7ic75Kc++rpiC78PT4x/4YVXTc71FVaPFPwnfKpIZ9rlio64iSEnRELDnLQpQe5k18tXHo9uhDwMbYmtbqkI8Ro2vuOA9L+6bg94f9W4pcLa7tXcXcd0ab4BH/3coLpR8qqlyvlBKEbvWcilj6+MHRpAxHV98/nAWawyfqBsXRoDL4a1o5monYy8E84kdzNIZ/cLkki1AVSRAkJq1gse04NjHLrnpJHRq8/sHE6D49AWh2BmgrMll2uFIyncRux5GUgJrDHxA7nr0HnBGIDJc8y55xupFrAwnroo0viIXsQ5Ix+6AUW9WW7iVqcf/0+S29X67HHEfBLOFoziPTaOK9VTbLEEN2jPdS0ZptDA4+KgVFYCqI/4sb93xphN8sRp/V9AP1xDSMGS83Apj7XsXauoS+0ezOIQAnWX44K/mwklmb3l2dBYHovQSa+tobBYLSoZ7awc+iso0C6+b7bn6vlEVdLjGgon8dG5cVO3lA5oTXoFGuNgaLRuQe17iXWJVBhIT9UHcVBUvvl5EN2zgrhHOxsgEn1581Drl/T5gaTvtcRg3fdJwit7Nx8zPDOSP8DznB6hh5wqSkF6eo+ug+7JJsCpwxvZ0ndA4/+YZChk258OF136tp9UMcRamvyNuIp/IPxEtWdIfp/L6b1MUVbfRiwR2zBdWQLK+txG1Rl2Hg3QvfnojRWn/who1ntBK X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1268; 6:6d3QbY/zc5EKnhVH27LU8Wfdm7zAupsHHiLgjMu9JxyHOghb1tnWWYCjethsNu34dDfVCO0h8SmJhTrsf8NV8TAUiL0jKZfroyRSodREAxFEmiunHHu80ySXYs9i14yEknU8xVkYBsRvQpWmrZ13crguWOUmk2iLqaZOTewqzALi0fHcxrjjhp7CW4R7Eh5hhjo/Ht+mWrJzAA3XELP0Rg49Rt019sUJeKe40ecx/4WmS5YIvJaL6txW33vQQXTYntgOQY6xbFHp/pvLE3PEl0MV4uQVYsyX5MeuSsB2A4IspH/tWDmovHOlixXOjjGrKMJAAWnIS3vnwSOpEFeVp1qk3GwwcxJL1FjhZX5PRi26/ZlgUdM8byHTuiSbkwXeSmpJuYGO4biTeNe6eiitsKtV0uw8NWEZINhVkzx/uMId6oV5Cn25v9kHoGXLivc3ZBjw4IZ23y/Z4vVNjP4O0toVCXYjAkW8lC6oM5af2Rrexyljnh3hU0NkV2WV45FeeQPhrfWSythDi2Ep8JFegw==; 5:NblnKpU0UG1fTXf7I9ZcAmok2t8iPJoWaJrWzs+7ZAvjedJa/1gCfxiDqcHmoB5T69ryPI0o5nA4mG4FJhBrX4Nj4Npv7qLs1G8wDpgsWX5AjnAJghBOO/5kycVPnsGOZCfGtQzoeuLZp/e7Y6klTA==; 24:s6PpVT2YnykW+rWLtUc4mhV/gk3lI0+s0z3yRx96vW1KsPhaPCyMxzDf0s4R3U5LM0+hf1gBvvnoEIFiPNnfWGBdn5o9W+7lHTHRG3oRBkk= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; BN3PR0701MB1268; 7:WDFm2M+xCseF6aLx9UeVrCLFwB5ekNdQRm8DIu5zsSS+568F4aP1ut6BKDtFuZH9aecV+bWN+LreFJrz/kDcPRJjgYLmB9nZIRmbPv4xhgMYy733oDVVxMLkX8bPRc2kxJv/tUlnsA0d0EykHeNKr2gTvDQ3F47YPYC2h7GGMzMWhlHBSNl0kAAoaQNPZytzh9axXRFKaip2RJYNXcD5/CeU1OkbaVEKjshZm9UHH/J9bpsQEu3bTZHkRCesRaeezD/lG/uroI9aCQDN4F5kQc5JGspBuhqYCQFQDk/k5a73viprZ9LhRBAAAhZFQaObwYQPyMavutmSgr3ArGkzlw== X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 May 2017 14:51:53.8221 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN3PR0701MB1268 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The patch 3 adds implementation for queued-based locking on ARM64, and the option in kernel config to enable it. Patches 1 and 2 fix some mess in header files to apply patch 3 smoothly. Tested on QDF2400 with huge improvements with these patches on the torture tests, by Adam Wallis. Tested on ThunderX, by Andrew Pinski: 120 thread (30 core - 4 thread/core) CN99xx (single socket): benchmark Units qspinlocks vs ticket locks sched/messaging s 73.91% sched/pipe ops/s 104.18% futex/hash ops/s 103.87% futex/wake ms 71.04% futex/wake-parallel ms 93.88% futex/requeue ms 96.47% futex/lock-pi ops/s 118.33% Notice, there's the queued locks implementation for the Power PC introduced by Pan Xinhui. He largely tested it and also found significant performance gain. In arch part it is very similar to this patch though. https://lwn.net/Articles/701137/ RFC: https://www.spinics.net/lists/arm-kernel/msg575575.html v1: - queued_spin_unlock_wait() and queued_spin_is_locked() are re-implemented in arch part to add additional memory barriers; - queued locks are made optional, ticket locks are enabled by default. Jan Glauber (1): arm64/locking: qspinlocks and qrwlocks support Yury Norov (2): kernel/locking: #include in qrwlock.c asm-generic: don't #include in qspinlock_types.h arch/arm64/Kconfig | 24 +++++++++++++++++++ arch/arm64/include/asm/qrwlock.h | 7 ++++++ arch/arm64/include/asm/qspinlock.h | 42 +++++++++++++++++++++++++++++++++ arch/arm64/include/asm/spinlock.h | 12 ++++++++++ arch/arm64/include/asm/spinlock_types.h | 14 ++++++++--- arch/arm64/kernel/Makefile | 1 + arch/arm64/kernel/qspinlock.c | 34 ++++++++++++++++++++++++++ include/asm-generic/qspinlock.h | 1 + include/asm-generic/qspinlock_types.h | 8 ------- kernel/locking/qrwlock.c | 1 + 10 files changed, 133 insertions(+), 11 deletions(-) create mode 100644 arch/arm64/include/asm/qrwlock.h create mode 100644 arch/arm64/include/asm/qspinlock.h create mode 100644 arch/arm64/kernel/qspinlock.c -- 2.11.0