From patchwork Mon Apr 28 04:56:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 885575 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 659AE19AD5C for ; Mon, 28 Apr 2025 04:56:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816175; cv=none; b=oSJdlCqpRkffqhGekzlukkt66JFJpFHYEzaS3Ug0IK1nY1CNPUljb7TIgSemJqpfn7nYsjC0HjM2ZiriTnQDgJIid3ScA1h8db5wQ75aoT3OktszgdSlIbvbLZtjA9J3FJvWYVmy9AdvAMPcV8BEtuUFZckz/oqBONWH6+8c5Y0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816175; c=relaxed/simple; bh=RD6ShG0gJARuG2fB+tFvdTnIY8OtoTZwykh1agSLJds=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To; b=j0IiwcEsGeie/vB4L/xoq0PJm6WWtcSuzmQVrjagXZ5iZqUkdehj+ulsKp+gmgmfNzasZanZrcBufRmt8t0ly7xy2TZcQy9nf1tgPYY/pk6BCvhis5T40lZ++wXUBZzMupppcoGNrVO0OXJNlrEbz7a7IrnBnhmLfLUYqZxCKtU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=PRzXhLLy; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="PRzXhLLy" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=To:Subject:From:References:In-Reply-To:Message-Id:Date:Sender: Reply-To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=YeqUax2no2iiWvm9VRoQd77K90ghmmQvP4J8eQ3/ReE=; b=PRzXhLLyaBD7ITo2mc/wi2hi1R 2IV9OueGkvdOFm1MdIvz3dxofqyD1j0LMjr32m1cWnxfZ9p0ImT1w9mo6Nf5RxVDyatbYv0FP3weg lvCWKNfi/ceIuUGvNJsutqacgoMgc4VXgP+6N+pSPeUVKhdeD5heGyc3KOS5D3A3ntYQYvsm3kpXq Va2HCM9hJbAREesA4K13Q4YqYDzT6C9C9C6ihzDWZKaYUeh88Kv0nrDt3KaHwkBzhVcqGVjIo19cH 2YS88hK8tzOnUUKO580k83uZsOKAXCx6CI5K1smgSdDYVRoYhtd03BrCuol2yUbAj8OLbHZzuXrap m3COoLPQ==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1u9GXH-001WEu-11; Mon, 28 Apr 2025 12:56:08 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Mon, 28 Apr 2025 12:56:07 +0800 Date: Mon, 28 Apr 2025 12:56:07 +0800 Message-Id: <5dac181449a282617176091f0d088f0fa1f2c0dd.1745815528.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 02/11] crypto: lib/poly1305 - Add block-only interface To: Linux Crypto Mailing List Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add a block-only interface for poly1305. Implement the generic code first. Also use the generic partial block helper. Signed-off-by: Herbert Xu --- include/crypto/internal/poly1305.h | 28 ++++++++++++++-- include/crypto/poly1305.h | 25 ++++++++++---- lib/crypto/poly1305.c | 54 +++++++++++++----------------- 3 files changed, 68 insertions(+), 39 deletions(-) diff --git a/include/crypto/internal/poly1305.h b/include/crypto/internal/poly1305.h index e614594f88c1..c60315f47562 100644 --- a/include/crypto/internal/poly1305.h +++ b/include/crypto/internal/poly1305.h @@ -6,9 +6,8 @@ #ifndef _CRYPTO_INTERNAL_POLY1305_H #define _CRYPTO_INTERNAL_POLY1305_H -#include -#include #include +#include /* * Poly1305 core functions. These only accept whole blocks; the caller must @@ -31,4 +30,29 @@ void poly1305_core_blocks(struct poly1305_state *state, void poly1305_core_emit(const struct poly1305_state *state, const u32 nonce[4], void *dst); +void poly1305_block_init_arch(struct poly1305_block_state *state, + const u8 raw_key[POLY1305_BLOCK_SIZE]); +void poly1305_block_init_generic(struct poly1305_block_state *state, + const u8 raw_key[POLY1305_BLOCK_SIZE]); +void poly1305_blocks_arch(struct poly1305_block_state *state, const u8 *src, + unsigned int len, u32 padbit); + +static inline void poly1305_blocks_generic(struct poly1305_block_state *state, + const u8 *src, unsigned int len, + u32 padbit) +{ + poly1305_core_blocks(&state->h, &state->core_r, src, + len / POLY1305_BLOCK_SIZE, padbit); +} + +void poly1305_emit_arch(const struct poly1305_state *state, + u8 digest[POLY1305_DIGEST_SIZE], const u32 nonce[4]); + +static inline void poly1305_emit_generic(const struct poly1305_state *state, + u8 digest[POLY1305_DIGEST_SIZE], + const u32 nonce[4]) +{ + poly1305_core_emit(state, nonce, digest); +} + #endif diff --git a/include/crypto/poly1305.h b/include/crypto/poly1305.h index 6e21ec2d1dc2..027d74842cd5 100644 --- a/include/crypto/poly1305.h +++ b/include/crypto/poly1305.h @@ -7,7 +7,6 @@ #define _CRYPTO_POLY1305_H #include -#include #define POLY1305_BLOCK_SIZE 16 #define POLY1305_KEY_SIZE 32 @@ -38,6 +37,17 @@ struct poly1305_state { }; }; +/* Combined state for block function. */ +struct poly1305_block_state { + /* accumulator */ + struct poly1305_state h; + /* key */ + union { + struct poly1305_key opaque_r[CONFIG_CRYPTO_LIB_POLY1305_RSIZE]; + struct poly1305_core_key core_r; + }; +}; + struct poly1305_desc_ctx { /* partial buffer */ u8 buf[POLY1305_BLOCK_SIZE]; @@ -45,12 +55,15 @@ struct poly1305_desc_ctx { unsigned int buflen; /* finalize key */ u32 s[4]; - /* accumulator */ - struct poly1305_state h; - /* key */ union { - struct poly1305_key opaque_r[CONFIG_CRYPTO_LIB_POLY1305_RSIZE]; - struct poly1305_core_key core_r; + struct { + struct poly1305_state h; + union { + struct poly1305_key opaque_r[CONFIG_CRYPTO_LIB_POLY1305_RSIZE]; + struct poly1305_core_key core_r; + }; + }; + struct poly1305_block_state state; }; }; diff --git a/lib/crypto/poly1305.c b/lib/crypto/poly1305.c index b633b043f0f6..9fec64a599c1 100644 --- a/lib/crypto/poly1305.c +++ b/lib/crypto/poly1305.c @@ -7,54 +7,45 @@ * Based on public domain code by Andrew Moon and Daniel J. Bernstein. */ +#include #include #include #include +#include #include +void poly1305_block_init_generic(struct poly1305_block_state *desc, + const u8 raw_key[POLY1305_BLOCK_SIZE]) +{ + poly1305_core_init(&desc->h); + poly1305_core_setkey(&desc->core_r, raw_key); +} +EXPORT_SYMBOL_GPL(poly1305_block_init_generic); + void poly1305_init_generic(struct poly1305_desc_ctx *desc, const u8 key[POLY1305_KEY_SIZE]) { - poly1305_core_setkey(&desc->core_r, key); desc->s[0] = get_unaligned_le32(key + 16); desc->s[1] = get_unaligned_le32(key + 20); desc->s[2] = get_unaligned_le32(key + 24); desc->s[3] = get_unaligned_le32(key + 28); - poly1305_core_init(&desc->h); desc->buflen = 0; + poly1305_block_init_generic(&desc->state, key); } EXPORT_SYMBOL_GPL(poly1305_init_generic); +static inline void poly1305_blocks(struct poly1305_block_state *state, + const u8 *src, unsigned int len) +{ + poly1305_blocks_generic(state, src, len, 1); +} + void poly1305_update_generic(struct poly1305_desc_ctx *desc, const u8 *src, unsigned int nbytes) { - unsigned int bytes; - - if (unlikely(desc->buflen)) { - bytes = min(nbytes, POLY1305_BLOCK_SIZE - desc->buflen); - memcpy(desc->buf + desc->buflen, src, bytes); - src += bytes; - nbytes -= bytes; - desc->buflen += bytes; - - if (desc->buflen == POLY1305_BLOCK_SIZE) { - poly1305_core_blocks(&desc->h, &desc->core_r, desc->buf, - 1, 1); - desc->buflen = 0; - } - } - - if (likely(nbytes >= POLY1305_BLOCK_SIZE)) { - poly1305_core_blocks(&desc->h, &desc->core_r, src, - nbytes / POLY1305_BLOCK_SIZE, 1); - src += nbytes - (nbytes % POLY1305_BLOCK_SIZE); - nbytes %= POLY1305_BLOCK_SIZE; - } - - if (unlikely(nbytes)) { - desc->buflen = nbytes; - memcpy(desc->buf, src, nbytes); - } + desc->buflen = BLOCK_HASH_UPDATE(poly1305_blocks, &desc->state, + src, nbytes, POLY1305_BLOCK_SIZE, + desc->buf, desc->buflen); } EXPORT_SYMBOL_GPL(poly1305_update_generic); @@ -64,10 +55,11 @@ void poly1305_final_generic(struct poly1305_desc_ctx *desc, u8 *dst) desc->buf[desc->buflen++] = 1; memset(desc->buf + desc->buflen, 0, POLY1305_BLOCK_SIZE - desc->buflen); - poly1305_core_blocks(&desc->h, &desc->core_r, desc->buf, 1, 0); + poly1305_blocks_generic(&desc->state, desc->buf, + POLY1305_BLOCK_SIZE, 0); } - poly1305_core_emit(&desc->h, desc->s, dst); + poly1305_emit_generic(&desc->h, dst, desc->s); *desc = (struct poly1305_desc_ctx){}; } EXPORT_SYMBOL_GPL(poly1305_final_generic); From patchwork Mon Apr 28 04:56:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 885574 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4B9F19AD5C for ; Mon, 28 Apr 2025 04:56:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816178; cv=none; b=LrTqKRjX5bGAbsUK9VsDiv9relsvCxHeCts6VHB7mQ/OhRq//cZHsWhU+UwfqXU4MA59R8JZxe0ly9mY6VI02bg8fJyPoPy4AfcgSf163XYLVJXmv90qn4BxuAFmgeVs4jLzjVKfxYhn7Fx4prz3Ba60cxDwtOueYUc+G3rB0eM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816178; c=relaxed/simple; bh=UuRt7OQ4Z7nSQ0BYTfR5ENiO02Moab2MSDIZMtxiitc=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To; b=fejmZHR/ETxVq1o1EGKo3lEJVoIWuep2fEe0rbCXdN6XlRnypQGa6gWMqH3tSn9SwJcuxVB8a3AcCvv+cPbURZOli2/j3dQcqdluz/17Ke+a1Wx2aFuD0mGCvsETfS1t4YmU3mPP0BgMQJLjSBEWqXcOZJPfF0lOfg873M/iQYA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=PAmLacsm; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="PAmLacsm" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=To:Subject:From:References:In-Reply-To:Message-Id:Date:Sender: Reply-To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=nMpmJ+1Dj0u63TbXpHYoBqzH0PHfnbRVekOhk4nMNMo=; b=PAmLacsmOZCVOtNf8t+ku3amzR PZkO/lNziOEsdSLiBu12AyvaRhwuoidiSDqyPMwenbl7jXoftEYk+5WdTGbbFGE7AH2wFsufg4xIT 38zUKPSpqtXzLPmiys+27iwsjCraVrlxvDk2CeKzO5VORAy5nfPvYKKnVICIQ56Z2p1u4l0YEZj68 aLPZNPhO+FoSHsQISYhg/QGEu6XuO+q8lRptCz2kOkmpkvG6ywQ5k38Z0OdK0CU5yQ/iNhYC98Ms8 k/ou8zamjHCCdWrH7wxdJe6FfoI/sclDwie0bhOD7AoRj4YTFw3RD0EwFbgeWHgCpBOqypFekZ9vw e5yE7P1g==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1u9GXL-001WFG-32; Mon, 28 Apr 2025 12:56:13 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Mon, 28 Apr 2025 12:56:11 +0800 Date: Mon, 28 Apr 2025 12:56:11 +0800 Message-Id: <09f6e4865da864bdffaae562107fd1a99c18c2c2.1745815528.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 04/11] crypto: arm64/poly1305 - Add block-only interface To: Linux Crypto Mailing List Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add block-only interface. Also remove the unnecessary SIMD fallback path. Signed-off-by: Herbert Xu --- arch/arm64/lib/crypto/Makefile | 3 +- arch/arm64/lib/crypto/poly1305-glue.c | 71 ++++++++++++++++----------- 2 files changed, 45 insertions(+), 29 deletions(-) diff --git a/arch/arm64/lib/crypto/Makefile b/arch/arm64/lib/crypto/Makefile index ac624c3effda..6207088397a7 100644 --- a/arch/arm64/lib/crypto/Makefile +++ b/arch/arm64/lib/crypto/Makefile @@ -5,7 +5,8 @@ chacha-neon-y := chacha-neon-core.o chacha-neon-glue.o obj-$(CONFIG_CRYPTO_POLY1305_NEON) += poly1305-neon.o poly1305-neon-y := poly1305-core.o poly1305-glue.o -AFLAGS_poly1305-core.o += -Dpoly1305_init=poly1305_init_arm64 +AFLAGS_poly1305-core.o += -Dpoly1305_init=poly1305_block_init_arch +AFLAGS_poly1305-core.o += -Dpoly1305_emit=poly1305_emit_arch quiet_cmd_perlasm = PERLASM $@ cmd_perlasm = $(PERL) $(<) void $(@) diff --git a/arch/arm64/lib/crypto/poly1305-glue.c b/arch/arm64/lib/crypto/poly1305-glue.c index 906970dd5373..d66a820e32d5 100644 --- a/arch/arm64/lib/crypto/poly1305-glue.c +++ b/arch/arm64/lib/crypto/poly1305-glue.c @@ -7,32 +7,60 @@ #include #include -#include -#include -#include +#include #include #include +#include #include +#include #include -asmlinkage void poly1305_init_arm64(void *state, const u8 *key); -asmlinkage void poly1305_blocks(void *state, const u8 *src, u32 len, u32 hibit); -asmlinkage void poly1305_blocks_neon(void *state, const u8 *src, u32 len, u32 hibit); -asmlinkage void poly1305_emit(void *state, u8 *digest, const u32 *nonce); +asmlinkage void poly1305_block_init_arch( + struct poly1305_block_state *state, + const u8 raw_key[POLY1305_BLOCK_SIZE]); +EXPORT_SYMBOL_GPL(poly1305_block_init_arch); +asmlinkage void poly1305_blocks(struct poly1305_block_state *state, + const u8 *src, u32 len, u32 hibit); +asmlinkage void poly1305_blocks_neon(struct poly1305_block_state *state, + const u8 *src, u32 len, u32 hibit); +asmlinkage void poly1305_emit_arch(const struct poly1305_state *state, + u8 digest[POLY1305_DIGEST_SIZE], + const u32 nonce[4]); +EXPORT_SYMBOL_GPL(poly1305_emit_arch); static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_neon); void poly1305_init_arch(struct poly1305_desc_ctx *dctx, const u8 key[POLY1305_KEY_SIZE]) { - poly1305_init_arm64(&dctx->h, key); dctx->s[0] = get_unaligned_le32(key + 16); dctx->s[1] = get_unaligned_le32(key + 20); dctx->s[2] = get_unaligned_le32(key + 24); dctx->s[3] = get_unaligned_le32(key + 28); dctx->buflen = 0; + poly1305_block_init_arch(&dctx->state, key); } EXPORT_SYMBOL(poly1305_init_arch); +void poly1305_blocks_arch(struct poly1305_block_state *state, const u8 *src, + unsigned int len, u32 padbit) +{ + len = round_down(len, POLY1305_BLOCK_SIZE); + if (static_branch_likely(&have_neon)) { + do { + unsigned int todo = min_t(unsigned int, len, SZ_4K); + + kernel_neon_begin(); + poly1305_blocks_neon(state, src, todo, 1); + kernel_neon_end(); + + len -= todo; + src += todo; + } while (len); + } else + poly1305_blocks(state, src, len, 1); +} +EXPORT_SYMBOL_GPL(poly1305_blocks_arch); + void poly1305_update_arch(struct poly1305_desc_ctx *dctx, const u8 *src, unsigned int nbytes) { @@ -45,29 +73,15 @@ void poly1305_update_arch(struct poly1305_desc_ctx *dctx, const u8 *src, dctx->buflen += bytes; if (dctx->buflen == POLY1305_BLOCK_SIZE) { - poly1305_blocks(&dctx->h, dctx->buf, POLY1305_BLOCK_SIZE, 1); + poly1305_blocks_arch(&dctx->state, dctx->buf, + POLY1305_BLOCK_SIZE, 1); dctx->buflen = 0; } } if (likely(nbytes >= POLY1305_BLOCK_SIZE)) { - unsigned int len = round_down(nbytes, POLY1305_BLOCK_SIZE); - - if (static_branch_likely(&have_neon) && crypto_simd_usable()) { - do { - unsigned int todo = min_t(unsigned int, len, SZ_4K); - - kernel_neon_begin(); - poly1305_blocks_neon(&dctx->h, src, todo, 1); - kernel_neon_end(); - - len -= todo; - src += todo; - } while (len); - } else { - poly1305_blocks(&dctx->h, src, len, 1); - src += len; - } + poly1305_blocks_arch(&dctx->state, src, nbytes, 1); + src += round_down(nbytes, POLY1305_BLOCK_SIZE); nbytes %= POLY1305_BLOCK_SIZE; } @@ -84,10 +98,11 @@ void poly1305_final_arch(struct poly1305_desc_ctx *dctx, u8 *dst) dctx->buf[dctx->buflen++] = 1; memset(dctx->buf + dctx->buflen, 0, POLY1305_BLOCK_SIZE - dctx->buflen); - poly1305_blocks(&dctx->h, dctx->buf, POLY1305_BLOCK_SIZE, 0); + poly1305_blocks_arch(&dctx->state, dctx->buf, + POLY1305_BLOCK_SIZE, 0); } - poly1305_emit(&dctx->h, dst, dctx->s); + poly1305_emit_arch(&dctx->h, dst, dctx->s); memzero_explicit(dctx, sizeof(*dctx)); } EXPORT_SYMBOL(poly1305_final_arch); From patchwork Mon Apr 28 04:56:16 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 885573 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 383F919AD5C for ; Mon, 28 Apr 2025 04:56:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816182; cv=none; b=Kl6zmoOygXiGN7PTiKFwFaNwJlzHabE9J6Y3VsUSWQ0UcDFfIrxGQT4GTNjXzIP+U5+i7DdcSMqwnBBw3iMa3EBEwR02IlIQTIlQ/LJy4BK1epqr+m7mz2hTRNfR02gJwtZR7SJlviqPU6jhiIxv8u0XxWT6T754A40vSONuh6U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816182; c=relaxed/simple; bh=ESx0J8pgoqbMFasL6vUSYb2D+iB33/ZG+uMlwsVXdXs=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To; b=UtCPBXSHjfw88l3FLamYRRpeL3/LEHpZiTSj3qbgL02AnwJwcDvMsw4l8+aSB76g7BlVOfxcEYDuhPzIlHk0biqq2lkEwohchSBmZg7OmhV2LdRmq7OgMh2ZcxuIWxydI6hdLRLy4fsStXtCB0xrB+7vw3ce98o25JJaA0wRfMQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=TxAuZHe5; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="TxAuZHe5" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=To:Subject:From:References:In-Reply-To:Message-Id:Date:Sender: Reply-To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=XV5PvwlJ4J+9DQkXbaYE7c06SZojO4Q/UNQTGMaL6kE=; b=TxAuZHe5o22qFt+9DII8ZvMBfD AklBq31R64hLrnMu7DFRU9nW3OkhHRs5O1FEMqr/lQc+f4TT5TRvZsK0Xa8AuKCpxG3hbr0WNiAbk 4VqHpPpEvgRZCcc7fzRuRt44Jo5LHmkLgTvH1Fh31nbTmmyxZhM9l9nC0sWiLCFyuYH3nxB2osQ2N kRRoDTyR6LmTG/J/P5/hNP7Q9N6RaetasBJoCyA1K5zSjE590KDrWZEUX6AHjLtXrbTlI5Cr5SHgO CiN17A3N80IQNvc+0mVFqd1bP+iVjCi3EMCKV9J6cMlBMIPGr471JV+OrjUF0RaEuVXY7A8vxjctn m+xPAG2Q==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1u9GXQ-001WFc-1q; Mon, 28 Apr 2025 12:56:17 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Mon, 28 Apr 2025 12:56:16 +0800 Date: Mon, 28 Apr 2025 12:56:16 +0800 Message-Id: <53b572d2695d6d554f365fca0c52f9f09fe7a8f8.1745815528.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 06/11] crypto: powerpc/poly1305 - Add block-only interface To: Linux Crypto Mailing List Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Add block-only interface. Also remove the unnecessary SIMD fallback path. Signed-off-by: Herbert Xu --- arch/powerpc/lib/crypto/poly1305-p10-glue.c | 84 ++++++++++++--------- 1 file changed, 49 insertions(+), 35 deletions(-) diff --git a/arch/powerpc/lib/crypto/poly1305-p10-glue.c b/arch/powerpc/lib/crypto/poly1305-p10-glue.c index 00617f4c58e6..708435beaba6 100644 --- a/arch/powerpc/lib/crypto/poly1305-p10-glue.c +++ b/arch/powerpc/lib/crypto/poly1305-p10-glue.c @@ -4,19 +4,20 @@ * * Copyright 2023- IBM Corp. All rights reserved. */ +#include +#include +#include +#include #include #include -#include -#include -#include -#include +#include #include -#include -#include -asmlinkage void poly1305_p10le_4blocks(void *h, const u8 *m, u32 mlen); -asmlinkage void poly1305_64s(void *h, const u8 *m, u32 mlen, int highbit); -asmlinkage void poly1305_emit_64(void *h, void *s, u8 *dst); +asmlinkage void poly1305_p10le_4blocks(struct poly1305_block_state *state, const u8 *m, u32 mlen); +asmlinkage void poly1305_64s(struct poly1305_block_state *state, const u8 *m, u32 mlen, int highbit); +asmlinkage void poly1305_emit_arch(const struct poly1305_state *state, + u8 digest[POLY1305_DIGEST_SIZE], + const u32 nonce[4]); static __ro_after_init DEFINE_STATIC_KEY_FALSE(have_p10); @@ -32,22 +33,49 @@ static void vsx_end(void) preempt_enable(); } -void poly1305_init_arch(struct poly1305_desc_ctx *dctx, const u8 key[POLY1305_KEY_SIZE]) +void poly1305_block_init_arch(struct poly1305_block_state *dctx, + const u8 raw_key[POLY1305_BLOCK_SIZE]) { if (!static_key_enabled(&have_p10)) - return poly1305_init_generic(dctx, key); + return poly1305_block_init_generic(dctx, raw_key); dctx->h = (struct poly1305_state){}; - dctx->core_r.key.r64[0] = get_unaligned_le64(key + 0); - dctx->core_r.key.r64[1] = get_unaligned_le64(key + 8); + dctx->core_r.key.r64[0] = get_unaligned_le64(raw_key + 0); + dctx->core_r.key.r64[1] = get_unaligned_le64(raw_key + 8); +} +EXPORT_SYMBOL_GPL(poly1305_block_init_arch); + +void poly1305_init_arch(struct poly1305_desc_ctx *dctx, const u8 key[POLY1305_KEY_SIZE]) +{ dctx->s[0] = get_unaligned_le32(key + 16); dctx->s[1] = get_unaligned_le32(key + 20); dctx->s[2] = get_unaligned_le32(key + 24); dctx->s[3] = get_unaligned_le32(key + 28); dctx->buflen = 0; + poly1305_block_init_arch(&dctx->state, key); } EXPORT_SYMBOL(poly1305_init_arch); +void poly1305_blocks_arch(struct poly1305_block_state *state, const u8 *src, + unsigned int len, u32 padbit) +{ + if (!static_key_enabled(&have_p10)) + return poly1305_blocks_generic(state, src, len, padbit); + vsx_begin(); + if (len >= POLY1305_BLOCK_SIZE * 4) { + poly1305_p10le_4blocks(state, src, len); + src += len - (len % (POLY1305_BLOCK_SIZE * 4)); + len %= POLY1305_BLOCK_SIZE * 4; + } + while (len >= POLY1305_BLOCK_SIZE) { + poly1305_64s(state, src, POLY1305_BLOCK_SIZE, padbit); + len -= POLY1305_BLOCK_SIZE; + src += POLY1305_BLOCK_SIZE; + } + vsx_end(); +} +EXPORT_SYMBOL_GPL(poly1305_blocks_arch); + void poly1305_update_arch(struct poly1305_desc_ctx *dctx, const u8 *src, unsigned int srclen) { @@ -64,28 +92,15 @@ void poly1305_update_arch(struct poly1305_desc_ctx *dctx, dctx->buflen += bytes; if (dctx->buflen < POLY1305_BLOCK_SIZE) return; - vsx_begin(); - poly1305_64s(&dctx->h, dctx->buf, POLY1305_BLOCK_SIZE, 1); - vsx_end(); + poly1305_blocks_arch(&dctx->state, dctx->buf, + POLY1305_BLOCK_SIZE, 1); dctx->buflen = 0; } if (likely(srclen >= POLY1305_BLOCK_SIZE)) { - bytes = round_down(srclen, POLY1305_BLOCK_SIZE); - if (crypto_simd_usable() && (srclen >= POLY1305_BLOCK_SIZE*4)) { - vsx_begin(); - poly1305_p10le_4blocks(&dctx->h, src, srclen); - vsx_end(); - src += srclen - (srclen % (POLY1305_BLOCK_SIZE * 4)); - srclen %= POLY1305_BLOCK_SIZE * 4; - } - while (srclen >= POLY1305_BLOCK_SIZE) { - vsx_begin(); - poly1305_64s(&dctx->h, src, POLY1305_BLOCK_SIZE, 1); - vsx_end(); - srclen -= POLY1305_BLOCK_SIZE; - src += POLY1305_BLOCK_SIZE; - } + poly1305_blocks_arch(&dctx->state, src, srclen, 1); + src += srclen - (srclen % POLY1305_BLOCK_SIZE); + srclen %= POLY1305_BLOCK_SIZE; } if (unlikely(srclen)) { @@ -104,12 +119,11 @@ void poly1305_final_arch(struct poly1305_desc_ctx *dctx, u8 *dst) dctx->buf[dctx->buflen++] = 1; memset(dctx->buf + dctx->buflen, 0, POLY1305_BLOCK_SIZE - dctx->buflen); - vsx_begin(); - poly1305_64s(&dctx->h, dctx->buf, POLY1305_BLOCK_SIZE, 0); - vsx_end(); + poly1305_blocks_arch(&dctx->state, dctx->buf, + POLY1305_BLOCK_SIZE, 0); } - poly1305_emit_64(&dctx->h, &dctx->s, dst); + poly1305_emit_arch(&dctx->h, dst, dctx->s); } EXPORT_SYMBOL(poly1305_final_arch); From patchwork Mon Apr 28 04:56:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 885572 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F369192B84 for ; Mon, 28 Apr 2025 04:56:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816187; cv=none; b=UTG6vhMrr34toB0U5805V7VCS5EsCJTrBm9GpqO2Vw60qekVzZViZtaght5PUl4qWTOPPkQKmkaK3UQwsqNgOgW7H8l1i5dA6kuEr/nI+6TpkKGMkRfdz2VE4jul+DFDcVqGmGy8XZoswsYzW7GOEfCiW+TpGOWRmL6OCGx0J94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816187; c=relaxed/simple; bh=mpQhYhnY/3FMu1IP0byDJFsigEkYqSBRBUECJhejk+E=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To; b=NdjGTjAf7Q+jBLUCFTFxeYWjFjQ6lyVDDeCLptDcZLqMjVrVWASWdCVmKLIIOs5TsQbCcW1QQQ3ML7bWQwVJyfGKN1NDVqwyZaz+VrujzldzJY0RGNvAU6iwZOIyzcV8eyFe6dnYFn797JEG+3Slt6EcVLmJq4+xYc3G7xK1Ae4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=lXoWdTx0; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="lXoWdTx0" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=To:Subject:From:References:In-Reply-To:Message-Id:Date:Sender: Reply-To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=0rog3zcGLNQ/vv+Rih0aLedD2DdiDVav3CaqUpzdBeM=; b=lXoWdTx0Dep2JqCA2xpY97H7Dx 0vRVDVn3jYsCXMxrzQZTqatpwL+IylbSBFrR3hoxh6eaUyaUQvDftMzB4oN1C//tQ59o+IemdQDsr kvv16QR1tGHSaOqOViCqnObSjG/2qjJFQyrn5F3y/DhRPPD9zYKjw88kmLArIf4Uq/bqT55Pfhc5J urOsSdIFVo24eEkBbRTGCNfq9UhBxXRZWt9XoqzzrIoIyx64A33oFyeqOYlpnXQSbgBgqRK/Xs1sb ornqV5ony6Deczu7169yEjRkHA3kHy5sD1DvvcZ28iJezFcQkXYKpvTzLKXOyBIkkA3S5fGHqmnJA bCyme6Aw==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1u9GXV-001WGM-0b; Mon, 28 Apr 2025 12:56:22 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Mon, 28 Apr 2025 12:56:21 +0800 Date: Mon, 28 Apr 2025 12:56:21 +0800 Message-Id: <0babdb56d14256b44249dc2bf3190ec200d9d738.1745815528.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 08/11] crypto: chacha20poly1305 - Use lib/crypto poly1305 To: Linux Crypto Mailing List Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Since the poly1305 algorithm is fixed, there is no point in going through the Crypto API for it. Use the lib/crypto poly1305 interface instead. For compatiblity keep the poly1305 parameter in the algorithm name. Signed-off-by: Herbert Xu --- crypto/Kconfig | 2 +- crypto/chacha20poly1305.c | 323 ++++++++------------------------------ 2 files changed, 67 insertions(+), 258 deletions(-) diff --git a/crypto/Kconfig b/crypto/Kconfig index 9878286d1d68..f87e2a26d2dd 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -784,8 +784,8 @@ config CRYPTO_AEGIS128_SIMD config CRYPTO_CHACHA20POLY1305 tristate "ChaCha20-Poly1305" select CRYPTO_CHACHA20 - select CRYPTO_POLY1305 select CRYPTO_AEAD + select CRYPTO_LIB_POLY1305 select CRYPTO_MANAGER help ChaCha20 stream cipher and Poly1305 authenticator combined diff --git a/crypto/chacha20poly1305.c b/crypto/chacha20poly1305.c index d740849f1c19..b29f66ba1e2f 100644 --- a/crypto/chacha20poly1305.c +++ b/crypto/chacha20poly1305.c @@ -12,36 +12,23 @@ #include #include #include -#include #include +#include #include +#include struct chachapoly_instance_ctx { struct crypto_skcipher_spawn chacha; - struct crypto_ahash_spawn poly; unsigned int saltlen; }; struct chachapoly_ctx { struct crypto_skcipher *chacha; - struct crypto_ahash *poly; /* key bytes we use for the ChaCha20 IV */ unsigned int saltlen; u8 salt[] __counted_by(saltlen); }; -struct poly_req { - /* zero byte padding for AD/ciphertext, as needed */ - u8 pad[POLY1305_BLOCK_SIZE]; - /* tail data with AD/ciphertext lengths */ - struct { - __le64 assoclen; - __le64 cryptlen; - } tail; - struct scatterlist src[1]; - struct ahash_request req; /* must be last member */ -}; - struct chacha_req { u8 iv[CHACHA_IV_SIZE]; struct scatterlist src[1]; @@ -62,7 +49,6 @@ struct chachapoly_req_ctx { /* request flags, with MAY_SLEEP cleared if needed */ u32 flags; union { - struct poly_req poly; struct chacha_req chacha; } u; }; @@ -105,16 +91,6 @@ static int poly_verify_tag(struct aead_request *req) return 0; } -static int poly_copy_tag(struct aead_request *req) -{ - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - - scatterwalk_map_and_copy(rctx->tag, req->dst, - req->assoclen + rctx->cryptlen, - sizeof(rctx->tag), 1); - return 0; -} - static void chacha_decrypt_done(void *data, int err) { async_done_continue(data, err, poly_verify_tag); @@ -151,210 +127,76 @@ static int chacha_decrypt(struct aead_request *req) return poly_verify_tag(req); } -static int poly_tail_continue(struct aead_request *req) +static int poly_hash(struct aead_request *req) { struct chachapoly_req_ctx *rctx = aead_request_ctx(req); + const void *zp = page_address(ZERO_PAGE(0)); + struct scatterlist *sg = req->src; + struct poly1305_desc_ctx desc; + struct scatter_walk walk; + struct { + union { + struct { + __le64 assoclen; + __le64 cryptlen; + }; + u8 u8[16]; + }; + } tail; + unsigned int padlen; + unsigned int total; + + if (sg != req->dst) + memcpy_sglist(req->dst, sg, req->assoclen); if (rctx->cryptlen == req->cryptlen) /* encrypting */ - return poly_copy_tag(req); + sg = req->dst; - return chacha_decrypt(req); -} + poly1305_init(&desc, rctx->key); + scatterwalk_start(&walk, sg); -static void poly_tail_done(void *data, int err) -{ - async_done_continue(data, err, poly_tail_continue); -} + total = rctx->assoclen; + while (total) { + unsigned int n = scatterwalk_next(&walk, total); -static int poly_tail(struct aead_request *req) -{ - struct crypto_aead *tfm = crypto_aead_reqtfm(req); - struct chachapoly_ctx *ctx = crypto_aead_ctx(tfm); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - int err; - - preq->tail.assoclen = cpu_to_le64(rctx->assoclen); - preq->tail.cryptlen = cpu_to_le64(rctx->cryptlen); - sg_init_one(preq->src, &preq->tail, sizeof(preq->tail)); - - ahash_request_set_callback(&preq->req, rctx->flags, - poly_tail_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - ahash_request_set_crypt(&preq->req, preq->src, - rctx->tag, sizeof(preq->tail)); - - err = crypto_ahash_finup(&preq->req); - if (err) - return err; - - return poly_tail_continue(req); -} - -static void poly_cipherpad_done(void *data, int err) -{ - async_done_continue(data, err, poly_tail); -} - -static int poly_cipherpad(struct aead_request *req) -{ - struct chachapoly_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - unsigned int padlen; - int err; - - padlen = -rctx->cryptlen % POLY1305_BLOCK_SIZE; - memset(preq->pad, 0, sizeof(preq->pad)); - sg_init_one(preq->src, preq->pad, padlen); - - ahash_request_set_callback(&preq->req, rctx->flags, - poly_cipherpad_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - ahash_request_set_crypt(&preq->req, preq->src, NULL, padlen); - - err = crypto_ahash_update(&preq->req); - if (err) - return err; - - return poly_tail(req); -} - -static void poly_cipher_done(void *data, int err) -{ - async_done_continue(data, err, poly_cipherpad); -} - -static int poly_cipher(struct aead_request *req) -{ - struct chachapoly_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - struct scatterlist *crypt = req->src; - int err; - - if (rctx->cryptlen == req->cryptlen) /* encrypting */ - crypt = req->dst; - - crypt = scatterwalk_ffwd(rctx->src, crypt, req->assoclen); - - ahash_request_set_callback(&preq->req, rctx->flags, - poly_cipher_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - ahash_request_set_crypt(&preq->req, crypt, NULL, rctx->cryptlen); - - err = crypto_ahash_update(&preq->req); - if (err) - return err; - - return poly_cipherpad(req); -} - -static void poly_adpad_done(void *data, int err) -{ - async_done_continue(data, err, poly_cipher); -} - -static int poly_adpad(struct aead_request *req) -{ - struct chachapoly_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - unsigned int padlen; - int err; + poly1305_update(&desc, walk.addr, n); + scatterwalk_done_src(&walk, n); + total -= n; + } padlen = -rctx->assoclen % POLY1305_BLOCK_SIZE; - memset(preq->pad, 0, sizeof(preq->pad)); - sg_init_one(preq->src, preq->pad, padlen); + poly1305_update(&desc, zp, padlen); - ahash_request_set_callback(&preq->req, rctx->flags, - poly_adpad_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - ahash_request_set_crypt(&preq->req, preq->src, NULL, padlen); + scatterwalk_skip(&walk, req->assoclen - rctx->assoclen); - err = crypto_ahash_update(&preq->req); - if (err) - return err; + total = rctx->cryptlen; + while (total) { + unsigned int n = scatterwalk_next(&walk, total); - return poly_cipher(req); -} + poly1305_update(&desc, walk.addr, n); + scatterwalk_done_src(&walk, n); + total -= n; + } -static void poly_ad_done(void *data, int err) -{ - async_done_continue(data, err, poly_adpad); -} + padlen = -rctx->cryptlen % POLY1305_BLOCK_SIZE; + poly1305_update(&desc, zp, padlen); -static int poly_ad(struct aead_request *req) -{ - struct chachapoly_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - int err; + tail.assoclen = cpu_to_le64(rctx->assoclen); + tail.cryptlen = cpu_to_le64(rctx->cryptlen); + poly1305_update(&desc, tail.u8, sizeof(tail)); + memzero_explicit(&tail, sizeof(tail)); + poly1305_final(&desc, rctx->tag); - ahash_request_set_callback(&preq->req, rctx->flags, - poly_ad_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - ahash_request_set_crypt(&preq->req, req->src, NULL, rctx->assoclen); + if (rctx->cryptlen != req->cryptlen) + return chacha_decrypt(req); - err = crypto_ahash_update(&preq->req); - if (err) - return err; - - return poly_adpad(req); -} - -static void poly_setkey_done(void *data, int err) -{ - async_done_continue(data, err, poly_ad); -} - -static int poly_setkey(struct aead_request *req) -{ - struct chachapoly_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - int err; - - sg_init_one(preq->src, rctx->key, sizeof(rctx->key)); - - ahash_request_set_callback(&preq->req, rctx->flags, - poly_setkey_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - ahash_request_set_crypt(&preq->req, preq->src, NULL, sizeof(rctx->key)); - - err = crypto_ahash_update(&preq->req); - if (err) - return err; - - return poly_ad(req); -} - -static void poly_init_done(void *data, int err) -{ - async_done_continue(data, err, poly_setkey); -} - -static int poly_init(struct aead_request *req) -{ - struct chachapoly_ctx *ctx = crypto_aead_ctx(crypto_aead_reqtfm(req)); - struct chachapoly_req_ctx *rctx = aead_request_ctx(req); - struct poly_req *preq = &rctx->u.poly; - int err; - - ahash_request_set_callback(&preq->req, rctx->flags, - poly_init_done, req); - ahash_request_set_tfm(&preq->req, ctx->poly); - - err = crypto_ahash_init(&preq->req); - if (err) - return err; - - return poly_setkey(req); + memcpy_to_scatterwalk(&walk, rctx->tag, sizeof(rctx->tag)); + return 0; } static void poly_genkey_done(void *data, int err) { - async_done_continue(data, err, poly_init); + async_done_continue(data, err, poly_hash); } static int poly_genkey(struct aead_request *req) @@ -388,7 +230,7 @@ static int poly_genkey(struct aead_request *req) if (err) return err; - return poly_init(req); + return poly_hash(req); } static void chacha_encrypt_done(void *data, int err) @@ -437,14 +279,7 @@ static int chachapoly_encrypt(struct aead_request *req) /* encrypt call chain: * - chacha_encrypt/done() * - poly_genkey/done() - * - poly_init/done() - * - poly_setkey/done() - * - poly_ad/done() - * - poly_adpad/done() - * - poly_cipher/done() - * - poly_cipherpad/done() - * - poly_tail/done/continue() - * - poly_copy_tag() + * - poly_hash() */ return chacha_encrypt(req); } @@ -458,13 +293,7 @@ static int chachapoly_decrypt(struct aead_request *req) /* decrypt call chain: * - poly_genkey/done() - * - poly_init/done() - * - poly_setkey/done() - * - poly_ad/done() - * - poly_adpad/done() - * - poly_cipher/done() - * - poly_cipherpad/done() - * - poly_tail/done/continue() + * - poly_hash() * - chacha_decrypt/done() * - poly_verify_tag() */ @@ -503,21 +332,13 @@ static int chachapoly_init(struct crypto_aead *tfm) struct chachapoly_instance_ctx *ictx = aead_instance_ctx(inst); struct chachapoly_ctx *ctx = crypto_aead_ctx(tfm); struct crypto_skcipher *chacha; - struct crypto_ahash *poly; unsigned long align; - poly = crypto_spawn_ahash(&ictx->poly); - if (IS_ERR(poly)) - return PTR_ERR(poly); - chacha = crypto_spawn_skcipher(&ictx->chacha); - if (IS_ERR(chacha)) { - crypto_free_ahash(poly); + if (IS_ERR(chacha)) return PTR_ERR(chacha); - } ctx->chacha = chacha; - ctx->poly = poly; ctx->saltlen = ictx->saltlen; align = crypto_aead_alignmask(tfm); @@ -525,12 +346,9 @@ static int chachapoly_init(struct crypto_aead *tfm) crypto_aead_set_reqsize( tfm, align + offsetof(struct chachapoly_req_ctx, u) + - max(offsetof(struct chacha_req, req) + - sizeof(struct skcipher_request) + - crypto_skcipher_reqsize(chacha), - offsetof(struct poly_req, req) + - sizeof(struct ahash_request) + - crypto_ahash_reqsize(poly))); + offsetof(struct chacha_req, req) + + sizeof(struct skcipher_request) + + crypto_skcipher_reqsize(chacha)); return 0; } @@ -539,7 +357,6 @@ static void chachapoly_exit(struct crypto_aead *tfm) { struct chachapoly_ctx *ctx = crypto_aead_ctx(tfm); - crypto_free_ahash(ctx->poly); crypto_free_skcipher(ctx->chacha); } @@ -548,7 +365,6 @@ static void chachapoly_free(struct aead_instance *inst) struct chachapoly_instance_ctx *ctx = aead_instance_ctx(inst); crypto_drop_skcipher(&ctx->chacha); - crypto_drop_ahash(&ctx->poly); kfree(inst); } @@ -559,7 +375,6 @@ static int chachapoly_create(struct crypto_template *tmpl, struct rtattr **tb, struct aead_instance *inst; struct chachapoly_instance_ctx *ctx; struct skcipher_alg_common *chacha; - struct hash_alg_common *poly; int err; if (ivsize > CHACHAPOLY_IV_SIZE) @@ -581,14 +396,9 @@ static int chachapoly_create(struct crypto_template *tmpl, struct rtattr **tb, goto err_free_inst; chacha = crypto_spawn_skcipher_alg_common(&ctx->chacha); - err = crypto_grab_ahash(&ctx->poly, aead_crypto_instance(inst), - crypto_attr_alg_name(tb[2]), 0, mask); - if (err) - goto err_free_inst; - poly = crypto_spawn_ahash_alg(&ctx->poly); - err = -EINVAL; - if (poly->digestsize != POLY1305_DIGEST_SIZE) + if (strcmp(crypto_attr_alg_name(tb[2]), "poly1305") && + strcmp(crypto_attr_alg_name(tb[2]), "poly1305-generic")) goto err_free_inst; /* Need 16-byte IV size, including Initial Block Counter value */ if (chacha->ivsize != CHACHA_IV_SIZE) @@ -599,16 +409,15 @@ static int chachapoly_create(struct crypto_template *tmpl, struct rtattr **tb, err = -ENAMETOOLONG; if (snprintf(inst->alg.base.cra_name, CRYPTO_MAX_ALG_NAME, - "%s(%s,%s)", name, chacha->base.cra_name, - poly->base.cra_name) >= CRYPTO_MAX_ALG_NAME) + "%s(%s,poly1305)", name, + chacha->base.cra_name) >= CRYPTO_MAX_ALG_NAME) goto err_free_inst; if (snprintf(inst->alg.base.cra_driver_name, CRYPTO_MAX_ALG_NAME, - "%s(%s,%s)", name, chacha->base.cra_driver_name, - poly->base.cra_driver_name) >= CRYPTO_MAX_ALG_NAME) + "%s(%s,poly1305-generic)", name, + chacha->base.cra_driver_name) >= CRYPTO_MAX_ALG_NAME) goto err_free_inst; - inst->alg.base.cra_priority = (chacha->base.cra_priority + - poly->base.cra_priority) / 2; + inst->alg.base.cra_priority = chacha->base.cra_priority; inst->alg.base.cra_blocksize = 1; inst->alg.base.cra_alignmask = chacha->base.cra_alignmask; inst->alg.base.cra_ctxsize = sizeof(struct chachapoly_ctx) + From patchwork Mon Apr 28 04:56:25 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Herbert Xu X-Patchwork-Id: 885571 Received: from abb.hmeau.com (abb.hmeau.com [144.6.53.87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8B9928E0F for ; Mon, 28 Apr 2025 04:56:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=144.6.53.87 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816191; cv=none; b=Z5fa9uisf19x7Fk1+J4inQEL8ct7ZUaI93PmVY7qojUoHNn7DMqttb+Xhs0oepl7IxCrmBcNs+NuddJgFt08l3wWi6RjGldivD5wmqjgoj6OkRtxi2qAxx33zaoXgHRzJP9yHtlLgL5Qlo+tTOt+Q0cfIfeO3YLmgdXOphuebo0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1745816191; c=relaxed/simple; bh=GGPLUur+XQbGCpfpOGxbeTjysautuHWthtv7hz5cZqI=; h=Date:Message-Id:In-Reply-To:References:From:Subject:To; b=Vbk4T9sash0N5upGz115J6pCfQ2eOA5mB/79E4DNHTwFyDaZ1IbCkwYx4ue1WHWpppqsKk61iXhaPNjCSPA4kbDA6CZ2I7hMfC5STef6OxD9kGWAXJmuSnn/JTg1+f9JtYJc5YaIW3D7x3vdb4FF5t+OS5EmWbVSwmWBzAd9Cms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au; spf=pass smtp.mailfrom=gondor.apana.org.au; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b=TKhhHaEI; arc=none smtp.client-ip=144.6.53.87 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gondor.apana.org.au Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=hmeau.com header.i=@hmeau.com header.b="TKhhHaEI" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=hmeau.com; s=formenos; h=To:Subject:From:References:In-Reply-To:Message-Id:Date:Sender: Reply-To:Cc:MIME-Version:Content-Type:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=+fFJOhU2X9/nYaSwK1s3nOIEjyF2K8vgL0+iXMOzn3A=; b=TKhhHaEIgBiA9MsQU1G18WYIcO 26AJ8RHEbsT/vBuv//T4O3JQJ2CX+NzjjsqSLsW2vY2obMouJ1XGt6jZfFFQWr8+OH5OITlNiDOHX No5kCqM3gdPfK9T/0Aw6UsjNS/sJKLCK/D1g7x1gQIHGkbDB7qejgqqBaWPEbn3ktxsrrc/xNxbr1 f2IjvU7fBFLHM5xro9KBCtF/Z0xKCEZHLk1LMjKxpnHWWLpyIqS7Xv851S9yGMGHnR3bj71q5W36T 1ani3ffAPZ7taspeEMm2TuVG5FL11VStU5D1Mi7pvYYSJJAXkqLm+gX0sMp37UlRKhjtq1z4y+8Bq UMC8PhNQ==; Received: from loth.rohan.me.apana.org.au ([192.168.167.2]) by formenos.hmeau.com with smtp (Exim 4.96 #2 (Debian)) id 1u9GXZ-001WH6-2R; Mon, 28 Apr 2025 12:56:26 +0800 Received: by loth.rohan.me.apana.org.au (sSMTP sendmail emulation); Mon, 28 Apr 2025 12:56:25 +0800 Date: Mon, 28 Apr 2025 12:56:25 +0800 Message-Id: <44211d959d0c77c69293f8e4d3d3b509446513d2.1745815528.git.herbert@gondor.apana.org.au> In-Reply-To: References: From: Herbert Xu Subject: [v4 PATCH 10/11] crypto: poly1305 - Remove algorithm To: Linux Crypto Mailing List Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: As there are no in-kernel users of the Crypto API poly1305 left, remove it. Signed-off-by: Herbert Xu --- crypto/Kconfig | 12 ---- crypto/Makefile | 2 - crypto/poly1305.c | 152 ---------------------------------------------- 3 files changed, 166 deletions(-) delete mode 100644 crypto/poly1305.c diff --git a/crypto/Kconfig b/crypto/Kconfig index f87e2a26d2dd..3cb5563dc4ab 100644 --- a/crypto/Kconfig +++ b/crypto/Kconfig @@ -953,18 +953,6 @@ config CRYPTO_POLYVAL This is used in HCTR2. It is not a general-purpose cryptographic hash function. -config CRYPTO_POLY1305 - tristate "Poly1305" - select CRYPTO_HASH - select CRYPTO_LIB_POLY1305 - select CRYPTO_LIB_POLY1305_GENERIC - help - Poly1305 authenticator algorithm (RFC7539) - - Poly1305 is an authenticator algorithm designed by Daniel J. Bernstein. - It is used for the ChaCha20-Poly1305 AEAD, specified in RFC7539 for use - in IETF protocols. This is the portable C implementation of Poly1305. - config CRYPTO_RMD160 tristate "RIPEMD-160" select CRYPTO_HASH diff --git a/crypto/Makefile b/crypto/Makefile index 5d2f2a28d8a0..587bc74b6d74 100644 --- a/crypto/Makefile +++ b/crypto/Makefile @@ -149,8 +149,6 @@ obj-$(CONFIG_CRYPTO_SEED) += seed.o obj-$(CONFIG_CRYPTO_ARIA) += aria_generic.o obj-$(CONFIG_CRYPTO_CHACHA20) += chacha.o CFLAGS_chacha.o += -DARCH=$(ARCH) -obj-$(CONFIG_CRYPTO_POLY1305) += poly1305.o -CFLAGS_poly1305.o += -DARCH=$(ARCH) obj-$(CONFIG_CRYPTO_DEFLATE) += deflate.o obj-$(CONFIG_CRYPTO_MICHAEL_MIC) += michael_mic.o obj-$(CONFIG_CRYPTO_CRC32C) += crc32c_generic.o diff --git a/crypto/poly1305.c b/crypto/poly1305.c deleted file mode 100644 index e0436bdc462b..000000000000 --- a/crypto/poly1305.c +++ /dev/null @@ -1,152 +0,0 @@ -/* - * Crypto API wrapper for the Poly1305 library functions - * - * Copyright (C) 2015 Martin Willi - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation; either version 2 of the License, or - * (at your option) any later version. - */ - -#include -#include -#include -#include -#include -#include - -struct crypto_poly1305_desc_ctx { - struct poly1305_desc_ctx base; - u8 key[POLY1305_KEY_SIZE]; - unsigned int keysize; -}; - -static int crypto_poly1305_init(struct shash_desc *desc) -{ - struct crypto_poly1305_desc_ctx *dctx = shash_desc_ctx(desc); - - dctx->keysize = 0; - return 0; -} - -static int crypto_poly1305_update(struct shash_desc *desc, - const u8 *src, unsigned int srclen, bool arch) -{ - struct crypto_poly1305_desc_ctx *dctx = shash_desc_ctx(desc); - unsigned int bytes; - - /* - * The key is passed as the first 32 "data" bytes. The actual - * poly1305_init() can be called only once the full key is available. - */ - if (dctx->keysize < POLY1305_KEY_SIZE) { - bytes = min(srclen, POLY1305_KEY_SIZE - dctx->keysize); - memcpy(&dctx->key[dctx->keysize], src, bytes); - dctx->keysize += bytes; - if (dctx->keysize < POLY1305_KEY_SIZE) - return 0; - if (arch) - poly1305_init(&dctx->base, dctx->key); - else - poly1305_init_generic(&dctx->base, dctx->key); - src += bytes; - srclen -= bytes; - } - - if (arch) - poly1305_update(&dctx->base, src, srclen); - else - poly1305_update_generic(&dctx->base, src, srclen); - - return 0; -} - -static int crypto_poly1305_update_generic(struct shash_desc *desc, - const u8 *src, unsigned int srclen) -{ - return crypto_poly1305_update(desc, src, srclen, false); -} - -static int crypto_poly1305_update_arch(struct shash_desc *desc, - const u8 *src, unsigned int srclen) -{ - return crypto_poly1305_update(desc, src, srclen, true); -} - -static int crypto_poly1305_final(struct shash_desc *desc, u8 *dst, bool arch) -{ - struct crypto_poly1305_desc_ctx *dctx = shash_desc_ctx(desc); - - if (unlikely(dctx->keysize != POLY1305_KEY_SIZE)) - return -ENOKEY; - - if (arch) - poly1305_final(&dctx->base, dst); - else - poly1305_final_generic(&dctx->base, dst); - memzero_explicit(&dctx->key, sizeof(dctx->key)); - return 0; -} - -static int crypto_poly1305_final_generic(struct shash_desc *desc, u8 *dst) -{ - return crypto_poly1305_final(desc, dst, false); -} - -static int crypto_poly1305_final_arch(struct shash_desc *desc, u8 *dst) -{ - return crypto_poly1305_final(desc, dst, true); -} - -static struct shash_alg poly1305_algs[] = { - { - .base.cra_name = "poly1305", - .base.cra_driver_name = "poly1305-generic", - .base.cra_priority = 100, - .base.cra_blocksize = POLY1305_BLOCK_SIZE, - .base.cra_module = THIS_MODULE, - .digestsize = POLY1305_DIGEST_SIZE, - .init = crypto_poly1305_init, - .update = crypto_poly1305_update_generic, - .final = crypto_poly1305_final_generic, - .descsize = sizeof(struct crypto_poly1305_desc_ctx), - }, - { - .base.cra_name = "poly1305", - .base.cra_driver_name = "poly1305-" __stringify(ARCH), - .base.cra_priority = 300, - .base.cra_blocksize = POLY1305_BLOCK_SIZE, - .base.cra_module = THIS_MODULE, - .digestsize = POLY1305_DIGEST_SIZE, - .init = crypto_poly1305_init, - .update = crypto_poly1305_update_arch, - .final = crypto_poly1305_final_arch, - .descsize = sizeof(struct crypto_poly1305_desc_ctx), - }, -}; - -static int num_algs; - -static int __init poly1305_mod_init(void) -{ - /* register the arch flavours only if they differ from generic */ - num_algs = poly1305_is_arch_optimized() ? 2 : 1; - - return crypto_register_shashes(poly1305_algs, num_algs); -} - -static void __exit poly1305_mod_exit(void) -{ - crypto_unregister_shashes(poly1305_algs, num_algs); -} - -subsys_initcall(poly1305_mod_init); -module_exit(poly1305_mod_exit); - -MODULE_LICENSE("GPL"); -MODULE_AUTHOR("Martin Willi "); -MODULE_DESCRIPTION("Crypto API wrapper for the Poly1305 library functions"); -MODULE_ALIAS_CRYPTO("poly1305"); -MODULE_ALIAS_CRYPTO("poly1305-generic"); -MODULE_ALIAS_CRYPTO("poly1305-" __stringify(ARCH));