From patchwork Thu Mar 27 11:29:00 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 27183 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f72.google.com (mail-qa0-f72.google.com [209.85.216.72]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 28E1220062 for ; Thu, 27 Mar 2014 11:28:45 +0000 (UTC) Received: by mail-qa0-f72.google.com with SMTP id f11sf6419038qae.3 for ; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:sender:precedence:list-id:x-original-sender :x-original-authentication-results:mailing-list:list-post:list-help :list-archive:list-unsubscribe; bh=m+nKXJVxXqJpeBFI464BNAFqjZstyD1QdYIEbS4izlM=; b=FTZ9SttfiUGgZrj2huXjjDkdwWPo2Ywz5jiNx8BiphrY+6/V62kJqMRlrGTJBvh0HF UxYDpL1m/N9RM4TsdagbktTLfK3abUfI/XJewH0p9B0fRL6VR7GLcZt7/OnV0+TYuxyH WfmJCQE1/nge+Mjo9/3+KmbKcxMB8kjgo6h+r1ZI02jlhDEgfNPEq7bj5shX0BMPnjGG iYcLd2WG0N+BPt8VNod+xf8dW2fPzPwemT2wQ/YVcOLbx8v0pb7Y/5RGVDTj0USAOys0 5M+IYZ99SYnh6LPF2VuUvYw9F7wrgkychWLaTO76vknRL4geEfbMbZ2cOGDiASqp/+P5 C1+g== X-Gm-Message-State: ALoCoQnl0ykC5Csz2svP70WpJ9UFsFt6KMvqltwynf9Ve5NZ1txzQpc1ZH9lesQ1hswfMM3u0nJV X-Received: by 10.52.103.103 with SMTP id fv7mr427127vdb.3.1395919724836; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.20.138 with SMTP id 10ls1007196qgj.92.gmail; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) X-Received: by 10.52.69.146 with SMTP id e18mr784083vdu.15.1395919724696; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) Received: from mail-vc0-f175.google.com (mail-vc0-f175.google.com [209.85.220.175]) by mx.google.com with ESMTPS id vd8si455927vdc.142.2014.03.27.04.28.44 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Mar 2014 04:28:44 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.220.175 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.220.175; Received: by mail-vc0-f175.google.com with SMTP id lh14so3976582vcb.34 for ; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) X-Received: by 10.221.37.1 with SMTP id tc1mr119334vcb.32.1395919724607; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp19733vck; Thu, 27 Mar 2014 04:28:44 -0700 (PDT) X-Received: by 10.68.249.100 with SMTP id yt4mr1114311pbc.165.1395919723843; Thu, 27 Mar 2014 04:28:43 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id bo2si1305997pbc.193.2014.03.27.04.28.43; Thu, 27 Mar 2014 04:28:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-crypto-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751914AbaC0L2m (ORCPT + 1 other); Thu, 27 Mar 2014 07:28:42 -0400 Received: from mail-we0-f177.google.com ([74.125.82.177]:62795 "EHLO mail-we0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751519AbaC0L2l (ORCPT ); Thu, 27 Mar 2014 07:28:41 -0400 Received: by mail-we0-f177.google.com with SMTP id u57so1687407wes.36 for ; Thu, 27 Mar 2014 04:28:40 -0700 (PDT) X-Received: by 10.180.77.49 with SMTP id p17mr39467924wiw.4.1395919720534; Thu, 27 Mar 2014 04:28:40 -0700 (PDT) Received: from ards-macbook-pro.local (cag06-7-83-153-85-71.fbx.proxad.net. [83.153.85.71]) by mx.google.com with ESMTPSA id w10sm11451675wiy.9.2014.03.27.04.28.39 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Mar 2014 04:28:39 -0700 (PDT) From: Ard Biesheuvel To: herbert@gondor.apana.org.au, linux-crypto@vger.kernel.org, x86@kernel.org, mingo@redhat.com, hpa@zytor.com Cc: Ard Biesheuvel Subject: [PATCH] x86/crypto: ghash: use C implementation for setkey() Date: Thu, 27 Mar 2014 12:29:00 +0100 Message-Id: <1395919740-20774-1-git-send-email-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 1.8.3.2 Sender: linux-crypto-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-crypto@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ard.biesheuvel@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.220.175 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , The GHASH setkey() function uses SSE registers but fails to call kernel_fpu_begin()/kernel_fpu_end(). Instead of adding these calls, and then having to deal with the restriction that they cannot be called from interrupt context, move the setkey() implementation to the C domain. Note that setkey() does not use any particular SSE features and is not expected to become a performance bottleneck. Signed-off-by: Ard Biesheuvel Acked-by: H. Peter Anvin --- I suppose this should be marked for stable as well? arch/x86/crypto/ghash-clmulni-intel_asm.S | 29 ----------------------------- arch/x86/crypto/ghash-clmulni-intel_glue.c | 12 +++++++++++- 2 files changed, 11 insertions(+), 30 deletions(-) diff --git a/arch/x86/crypto/ghash-clmulni-intel_asm.S b/arch/x86/crypto/ghash-clmulni-intel_asm.S index 586f41aac361..185fad49d86f 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_asm.S +++ b/arch/x86/crypto/ghash-clmulni-intel_asm.S @@ -24,10 +24,6 @@ .align 16 .Lbswap_mask: .octa 0x000102030405060708090a0b0c0d0e0f -.Lpoly: - .octa 0xc2000000000000000000000000000001 -.Ltwo_one: - .octa 0x00000001000000000000000000000001 #define DATA %xmm0 #define SHASH %xmm1 @@ -134,28 +130,3 @@ ENTRY(clmul_ghash_update) .Lupdate_just_ret: ret ENDPROC(clmul_ghash_update) - -/* - * void clmul_ghash_setkey(be128 *shash, const u8 *key); - * - * Calculate hash_key << 1 mod poly - */ -ENTRY(clmul_ghash_setkey) - movaps .Lbswap_mask, BSWAP - movups (%rsi), %xmm0 - PSHUFB_XMM BSWAP %xmm0 - movaps %xmm0, %xmm1 - psllq $1, %xmm0 - psrlq $63, %xmm1 - movaps %xmm1, %xmm2 - pslldq $8, %xmm1 - psrldq $8, %xmm2 - por %xmm1, %xmm0 - # reduction - pshufd $0b00100100, %xmm2, %xmm1 - pcmpeqd .Ltwo_one, %xmm1 - pand .Lpoly, %xmm1 - pxor %xmm1, %xmm0 - movups %xmm0, (%rdi) - ret -ENDPROC(clmul_ghash_setkey) diff --git a/arch/x86/crypto/ghash-clmulni-intel_glue.c b/arch/x86/crypto/ghash-clmulni-intel_glue.c index 6759dd1135be..3945d8095e80 100644 --- a/arch/x86/crypto/ghash-clmulni-intel_glue.c +++ b/arch/x86/crypto/ghash-clmulni-intel_glue.c @@ -58,13 +58,23 @@ static int ghash_setkey(struct crypto_shash *tfm, const u8 *key, unsigned int keylen) { struct ghash_ctx *ctx = crypto_shash_ctx(tfm); + be128 *x = (be128 *)key; + u64 a, b; if (keylen != GHASH_BLOCK_SIZE) { crypto_shash_set_flags(tfm, CRYPTO_TFM_RES_BAD_KEY_LEN); return -EINVAL; } - clmul_ghash_setkey(&ctx->shash, key); + /* perform multiplication by 'x' in GF(2^128) */ + a = be64_to_cpu(x->a); + b = be64_to_cpu(x->b); + + ctx->shash.a = (__be64)((b << 1) ^ (a >> 63)); + ctx->shash.b = (__be64)((a << 1) | (b >> 63)); + + if (a >> 63) + ctx->shash.b ^= cpu_to_be64(0xc2); return 0; }