From patchwork Sat Jun 27 21:17:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216974 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 332D5C433E1 for ; Sat, 27 Jun 2020 21:18:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 13D34206D7 for ; Sat, 27 Jun 2020 21:18:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="UNM+fr7C" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726453AbgF0VSv (ORCPT ); Sat, 27 Jun 2020 17:18:51 -0400 Received: from mail-eopbgr60076.outbound.protection.outlook.com ([40.107.6.76]:16519 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725907AbgF0VSt (ORCPT ); Sat, 27 Jun 2020 17:18:49 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Fo3v8kBB0BnGl3i8gjg9ajhBfbh90uJu6ffVZa2YlWYlUF693tG1JIwXr4LaGMhWgrWVoYmwEDCxJYu6A8NpxhUbOfSOGq7/nmsK1/JKq/YBngOqILhMAGO/wm9B86CWD2GOVYF0R6rAQxqx9vb8OCuN6yUMZ5Vnn5R/jrYYH8cIV/IkhBpPR5cqjPAZ2DYUheTMlYP2rTnENDRUTTpmQ2F8ZvYDeg4oIPjf9RJe6TbtHDRuz72EuZPdLFx9waVALjOh4MztoaY7uI0aJg7ArmcEcVYE9yvNWIEj+0jFEP2BxzXpwf2zuKHLfR1I8LdMFF/aLSTtxtQx2kUcZsQbRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YcTKo9GB/DW4Frmyq26xfVP4s5H3QDZUecc8m679fYk=; b=LnKFj05jSK80llnTmloGChOnHOAoeVcUyeKcT+GHy388akjtygvz4sT0WXpWjIU0qUw3rJ1Pqo1j08CjgcEgGPVMsFtovIAjO4C6AU0dNnxJUq6ZpI+asTfF5oJ4hxwxRVE4CO1c6FZxUNp0g0rqTDqEYFh7BGO1bEsQxbmqQhkoCvcdc7Yh/ORfnkHi8ZrXej7O38N196QkcVBXkdI+Eaxi9Hkct3BrPYbjHIa0ZeY1S4EUo7kP9/NaK97Oom1ZDQK6HLNhl2qzsKgDmz8b5w3tVcG28OKWY0opyisLx++g80v0W0tB1TPyGkwqFVeN9QZGo3J1Efgv8uS/LJXung== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YcTKo9GB/DW4Frmyq26xfVP4s5H3QDZUecc8m679fYk=; b=UNM+fr7CbTRQnOpsd4cCN4rG+zLiLKVGd+ElAL6FEjNhKef33E7v8ZfVJh9n2pPPN/Q86jkJaaQztugf1gqoQ8qolYC+lulpmC2e6DYZjKZCwT08Dux3/2nI0oHlKol3Jy1/bg2VyWajTv68xL4iPqRPV56k0kgx2tau5vdT1fU= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:18:39 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:18:39 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Tariq Toukan , Maxim Mikityanskiy , Saeed Mahameed Subject: [net-next 02/15] net/mlx5e: Refactor build channel params Date: Sat, 27 Jun 2020 14:17:14 -0700 Message-Id: <20200627211727.259569-3-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:18:37 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 927642dc-6938-42e9-dd1a-08d81adfa957 X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3044; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: kAVfQF0CCILXpJrAqoKok8f1agjQ0a3nINjY+6o7DOh70K3+wJqzYWudiPnBDtIC0niXkZmx3pIl3xJn+TPs04JcM8uh6q3USOuOw7/InNOtTTH7o4ePmYIhSylBkKx3r2WtZbhqJtn7IZDHOttQf+wCBbmE080bM8/gGycI04udoIDBGxVRj+RLAw5WASOU3kzEmC7c7P1UDbjfUofEFRG8AJ6f/kt24W/ikcGpRehZCt8gRmRvt2xd3DQauIE1m1NudkgB6CzsVOL4/18tZDPGohdQUMU2Z9x+dVNjMAHOI6bOevkd5e50rgG5ioC5w7Pu7SBmMLvxJ8VBXxGZZs2UVkSc4+In/RORDy0l+f03JrIx8N57YcNPm8YC5XKQ X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 5U5VBsi/ryGxA/Z5w4IeVvRTsGZvwB1qAVCkKkevwXleKRcp9yN6wNXPn0qHstuc3Kq4hOzHiq7ZY3pv5132Vmq8J5i471Go6yucOLWq+ODrzPYTjy5/8YyPOMkYOY4tnj6SrWhO/auUoMfkOohA1WPGAItUjNPM6ktZENmw70W/8cNOHgPBderqEnWMAWERDxIIRdOlWjjizqzb7khkWIPn6zEEtG9eiOghFd6bxkLyyaolhCtR12mkoT5hLFsC8d4RsUvJDcc6tI73iyMTwPGVU6eK1VpLDJBmJ4mjPATaa0yyvpNQzCoaITSPlF1l74xzD2dXCeuWQseawYj8yOXfpQhZ+DgNns/PGumqy8R+hks+CtU3vtATccEhtMvE+pWIdfZgwmPettH5OynYO3uspjU9CQ4PNQ/VzxYMQj0gdOnWcKj96MRtD5SujmBsV1cM8UMSVCXEbiPlUMZpqfPIzjIcD/5dz/twzWQy62w= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 927642dc-6938-42e9-dd1a-08d81adfa957 X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:18:39.6031 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: H+oOOwpjBJykLOgjuV2b17OIgjMxMkW9w3wDsED8Lz5wH4aa9pV9k9VyKEG7SFuzydfX0c3osTQQwXRt2321Ew== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tariq Toukan Take the CQ params into their respective RQ/SQ params. Split the params build of the different ICOSQs (sync and async), as they require different init values. Signed-off-by: Tariq Toukan Reviewed-by: Maxim Mikityanskiy Signed-off-by: Saeed Mahameed --- .../ethernet/mellanox/mlx5/core/en/params.h | 22 +++++++------- .../mellanox/mlx5/core/en/xsk/setup.c | 6 ++-- .../net/ethernet/mellanox/mlx5/core/en_main.c | 30 ++++++++++--------- 3 files changed, 29 insertions(+), 29 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h index 989d8f429438..a87273e801b2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/params.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/params.h @@ -11,33 +11,33 @@ struct mlx5e_xsk_param { u16 chunk_size; }; +struct mlx5e_cq_param { + u32 cqc[MLX5_ST_SZ_DW(cqc)]; + struct mlx5_wq_param wq; + u16 eq_ix; + u8 cq_period_mode; +}; + struct mlx5e_rq_param { + struct mlx5e_cq_param cqp; u32 rqc[MLX5_ST_SZ_DW(rqc)]; struct mlx5_wq_param wq; struct mlx5e_rq_frags_info frags_info; }; struct mlx5e_sq_param { + struct mlx5e_cq_param cqp; u32 sqc[MLX5_ST_SZ_DW(sqc)]; struct mlx5_wq_param wq; bool is_mpw; }; -struct mlx5e_cq_param { - u32 cqc[MLX5_ST_SZ_DW(cqc)]; - struct mlx5_wq_param wq; - u16 eq_ix; - u8 cq_period_mode; -}; - struct mlx5e_channel_param { struct mlx5e_rq_param rq; - struct mlx5e_sq_param sq; + struct mlx5e_sq_param txq_sq; struct mlx5e_sq_param xdp_sq; struct mlx5e_sq_param icosq; - struct mlx5e_cq_param rx_cq; - struct mlx5e_cq_param tx_cq; - struct mlx5e_cq_param icosq_cq; + struct mlx5e_sq_param async_icosq; }; static inline bool mlx5e_qid_get_ch_if_in_group(struct mlx5e_params *params, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c index 1eb817e62830..cc46414773b5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c @@ -41,8 +41,6 @@ static void mlx5e_build_xsk_cparam(struct mlx5e_priv *priv, { mlx5e_build_rq_param(priv, params, xsk, &cparam->rq); mlx5e_build_xdpsq_param(priv, params, &cparam->xdp_sq); - mlx5e_build_rx_cq_param(priv, params, xsk, &cparam->rx_cq); - mlx5e_build_tx_cq_param(priv, params, &cparam->tx_cq); } int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params, @@ -61,7 +59,7 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params, mlx5e_build_xsk_cparam(priv, params, xsk, cparam); - err = mlx5e_open_cq(c, params->rx_cq_moderation, &cparam->rx_cq, &c->xskrq.cq); + err = mlx5e_open_cq(c, params->rx_cq_moderation, &cparam->rq.cqp, &c->xskrq.cq); if (unlikely(err)) goto err_free_cparam; @@ -69,7 +67,7 @@ int mlx5e_open_xsk(struct mlx5e_priv *priv, struct mlx5e_params *params, if (unlikely(err)) goto err_close_rx_cq; - err = mlx5e_open_cq(c, params->tx_cq_moderation, &cparam->tx_cq, &c->xsksq.cq); + err = mlx5e_open_cq(c, params->tx_cq_moderation, &cparam->xdp_sq.cqp, &c->xsksq.cq); if (unlikely(err)) goto err_close_rq; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 72ce5808d583..11997c23dfb5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -1675,7 +1675,7 @@ static int mlx5e_open_tx_cqs(struct mlx5e_channel *c, for (tc = 0; tc < c->num_tc; tc++) { err = mlx5e_open_cq(c, params->tx_cq_moderation, - &cparam->tx_cq, &c->sq[tc].cq); + &cparam->txq_sq.cqp, &c->sq[tc].cq); if (err) goto err_close_tx_cqs; } @@ -1707,7 +1707,7 @@ static int mlx5e_open_sqs(struct mlx5e_channel *c, int txq_ix = c->ix + tc * params->num_channels; err = mlx5e_open_txqsq(c, c->priv->tisn[c->lag_port][tc], txq_ix, - params, &cparam->sq, &c->sq[tc], tc); + params, &cparam->txq_sq, &c->sq[tc], tc); if (err) goto err_close_sqs; } @@ -1817,11 +1817,11 @@ static int mlx5e_open_queues(struct mlx5e_channel *c, struct dim_cq_moder icocq_moder = {0, 0}; int err; - err = mlx5e_open_cq(c, icocq_moder, &cparam->icosq_cq, &c->async_icosq.cq); + err = mlx5e_open_cq(c, icocq_moder, &cparam->icosq.cqp, &c->async_icosq.cq); if (err) return err; - err = mlx5e_open_cq(c, icocq_moder, &cparam->icosq_cq, &c->icosq.cq); + err = mlx5e_open_cq(c, icocq_moder, &cparam->async_icosq.cqp, &c->icosq.cq); if (err) goto err_close_async_icosq_cq; @@ -1829,17 +1829,16 @@ static int mlx5e_open_queues(struct mlx5e_channel *c, if (err) goto err_close_icosq_cq; - err = mlx5e_open_cq(c, params->tx_cq_moderation, &cparam->tx_cq, &c->xdpsq.cq); + err = mlx5e_open_cq(c, params->tx_cq_moderation, &cparam->xdp_sq.cqp, &c->xdpsq.cq); if (err) goto err_close_tx_cqs; - err = mlx5e_open_cq(c, params->rx_cq_moderation, &cparam->rx_cq, &c->rq.cq); + err = mlx5e_open_cq(c, params->rx_cq_moderation, &cparam->rq.cqp, &c->rq.cq); if (err) goto err_close_xdp_tx_cqs; - /* XDP SQ CQ params are same as normal TXQ sq CQ params */ err = c->xdp ? mlx5e_open_cq(c, params->tx_cq_moderation, - &cparam->tx_cq, &c->rq_xdpsq.cq) : 0; + &cparam->xdp_sq.cqp, &c->rq_xdpsq.cq) : 0; if (err) goto err_close_rx_cq; @@ -1847,7 +1846,7 @@ static int mlx5e_open_queues(struct mlx5e_channel *c, spin_lock_init(&c->async_icosq_lock); - err = mlx5e_open_icosq(c, params, &cparam->icosq, &c->async_icosq); + err = mlx5e_open_icosq(c, params, &cparam->async_icosq, &c->async_icosq); if (err) goto err_disable_napi; @@ -2158,6 +2157,7 @@ void mlx5e_build_rq_param(struct mlx5e_priv *priv, MLX5_SET(rqc, rqc, scatter_fcs, params->scatter_fcs_en); param->wq.buf_numa_node = dev_to_node(mdev->device); + mlx5e_build_rx_cq_param(priv, params, xsk, ¶m->cqp); } static void mlx5e_build_drop_rq_param(struct mlx5e_priv *priv, @@ -2200,6 +2200,7 @@ static void mlx5e_build_sq_param(struct mlx5e_priv *priv, mlx5e_build_sq_param_common(priv, param); MLX5_SET(wq, wq, log_wq_sz, params->log_sq_size); MLX5_SET(sqc, sqc, allow_swp, allow_swp); + mlx5e_build_tx_cq_param(priv, params, ¶m->cqp); } static void mlx5e_build_common_cq_param(struct mlx5e_priv *priv, @@ -2276,6 +2277,7 @@ void mlx5e_build_icosq_param(struct mlx5e_priv *priv, MLX5_SET(wq, wq, log_wq_sz, log_wq_size); MLX5_SET(sqc, sqc, reg_umr, MLX5_CAP_ETH(priv->mdev, reg_umr_sq)); + mlx5e_build_ico_cq_param(priv, log_wq_size, ¶m->cqp); } void mlx5e_build_xdpsq_param(struct mlx5e_priv *priv, @@ -2288,6 +2290,7 @@ void mlx5e_build_xdpsq_param(struct mlx5e_priv *priv, mlx5e_build_sq_param_common(priv, param); MLX5_SET(wq, wq, log_wq_sz, params->log_sq_size); param->is_mpw = MLX5E_GET_PFLAG(params, MLX5E_PFLAG_XDP_TX_MPWQE); + mlx5e_build_tx_cq_param(priv, params, ¶m->cqp); } static u8 mlx5e_build_icosq_log_wq_sz(struct mlx5e_params *params, @@ -2306,18 +2309,17 @@ static void mlx5e_build_channel_param(struct mlx5e_priv *priv, struct mlx5e_params *params, struct mlx5e_channel_param *cparam) { - u8 icosq_log_wq_sz; + u8 icosq_log_wq_sz, async_icosq_log_wq_sz; mlx5e_build_rq_param(priv, params, NULL, &cparam->rq); icosq_log_wq_sz = mlx5e_build_icosq_log_wq_sz(params, &cparam->rq); + async_icosq_log_wq_sz = MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE; - mlx5e_build_sq_param(priv, params, &cparam->sq); + mlx5e_build_sq_param(priv, params, &cparam->txq_sq); mlx5e_build_xdpsq_param(priv, params, &cparam->xdp_sq); mlx5e_build_icosq_param(priv, icosq_log_wq_sz, &cparam->icosq); - mlx5e_build_rx_cq_param(priv, params, NULL, &cparam->rx_cq); - mlx5e_build_tx_cq_param(priv, params, &cparam->tx_cq); - mlx5e_build_ico_cq_param(priv, icosq_log_wq_sz, &cparam->icosq_cq); + mlx5e_build_icosq_param(priv, async_icosq_log_wq_sz, &cparam->async_icosq); } int mlx5e_open_channels(struct mlx5e_priv *priv, From patchwork Sat Jun 27 21:17:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E0A35C433DF for ; Sat, 27 Jun 2020 21:19:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BC16220738 for ; Sat, 27 Jun 2020 21:19:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="DLcRrG0d" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726608AbgF0VS6 (ORCPT ); Sat, 27 Jun 2020 17:18:58 -0400 Received: from mail-eopbgr60076.outbound.protection.outlook.com ([40.107.6.76]:16519 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725907AbgF0VS4 (ORCPT ); Sat, 27 Jun 2020 17:18:56 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J+F7ppoNs5YWIooBLn+iPzmQGE5AoyAbSiloAL978TpdBL6sTRAGLHf9bV8vJdlGG96SaCWjLUGSavIyRE8KM6a4dQRVzsBW4dZrHrgqxP4Pjm3YoHF0XToHLTYjM5lWY3dXMn1iBVBxwOrkzETt5BXjcY6TeuFIHgIeHqwkZ7PMzNFwbznq+XuMP5aaZX4RkpQaUsTrFINPpzWiRW4Gn04kv6hmH4Jz7/GmjizeCpl38p6qgN36xZNhWfrBQTqtXPVS7m34mDQelm18pzqvlJqwygUPWRWFmqZc+NpalPBY5jZjXKGGN3XcfPLCxPkVpnazDYHOMEYHLWz0McygCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K5dSNaot+DwlnQLjnePK9OGTL37zZEONp+wD0plwSe8=; b=gdzyTQzpi5ERzT+mPBsEzWJi25Vst/IIPu5WIVD3DbMx1DgjjE8ISZX+nlOE2kOt/CpXsAWzvfTvzdIDL/8cyx2huEyAL2TQlIphvbhN5aQdEY9RFkoUWP2K7Rg9U/ITtWaxPNlMsyuhIHZwT2mY12Tomkv+NQ3/On4CPRgHxiAN9CqQoErlV+XjxomXZZx/401a2uz4DE2vPkHJYHpf6vrgdLBbOrqpizoOpNk1NPqHOdyniUdOIbiGzWozQeklGcSuQjlblGuVddJevpHJ7cRpZ7nWsaJh6Y5A7rwx9nBt72Ofn50gJ6NYBhTel3I0SgfiDcEH51/q7/noYXRksw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K5dSNaot+DwlnQLjnePK9OGTL37zZEONp+wD0plwSe8=; b=DLcRrG0dDRGzj66FEHiu+lWcL8+9RPjsy6AQ4yaIcNkpKTTUDOF/tz9lGiL50gzfSJbLxzTzbXo3qtcU3iJlAjs2VpOS9APG/AEAPzMdIVZGnox/Tbz0CQkKWM1u6KcT0wrQi48Z8JiA+kAxClaOv3WR5Xluz+SzptCexEZcuL0= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:18:44 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:18:44 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Boris Pismenny , Tariq Toukan , Saeed Mahameed Subject: [net-next 04/15] net/mlx5e: Receive flow steering framework for accelerated TCP flows Date: Sat, 27 Jun 2020 14:17:16 -0700 Message-Id: <20200627211727.259569-5-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:18:42 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 934f7be8-d8fe-41ee-d1cf-08d81adfac17 X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:530; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fQCY3cDeZF6tEOFGEC278oMvivFk0Uki1SOaWe+wfRnVnSyS+PusjXUhGGrtdgMAXIyzQp23VEZ4HiN5TiPjhioVw8gqQYEajhOnm3KWWCGZP2J/U3NPTvC8QRiCA0XTwXOO8ePTG04Ggf8Uuo037Dj93b3HqsCZqtffL/7LnhVvPk5U6OuPQunmQgz+63sUqsrk05nyhA6Flu+jO+yd3zp7AfGMN+hAaycZZKbL6RLPObFXR2hNaBwzjh/+0SJ2Tw4+FTsNOCMWMoqQLcO4rb5Id0nV/DcmH6p8WOtKCXIPzGjQ0oEykohO38zBJkSpioOBvdfPoNAFc4ca9mw4aibwhFgYQu03KELMeslRePaG7NkIFGfBjaKr4+Rc8x8W X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(30864003)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: veK3PP8fJ01Xb1rABCQ7GG7ctDAMGOFM296Xb2+DegwrqNDESmGpsX8wW+Veuajxu9tRW+tePuNhK7UVm9Pej6REnu9Ysp7FD6BebzVpX5ilXTxeQUVSmoYt82JGE0uxPhF7r31WVm+2LCSQ/LgAzH0Ity/sTVupj6ZSPVT0yxz0yg0579RWnDWnPr4zVr1hgN/rnfi+9RQTZmWtrb+dhHhkkMZIvaL+WN3Pf3g6kTl84lDHAQ//XtMmHvodeuWCW59WoErscmgu5Xu0Xpvb1fOC4cYTAUtUYD66P76L2z1DjBX3TbpYMWpaCxmuPSZ90/E/7esd1INPyLeYX3v9f4QMKwfXqIKzePWUgTqWyT7+NamZz3D+TBFL1jWQIe9WjcunKJDQeaJ4C8+0G7W/CZmcyFwQ362/kivaDuDMnppRMHoliOfib+K0DFtkI6ibU42xrrHKx9kaHIsoNwiq9OScnkzViOaABwtVqj+AH+8= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 934f7be8-d8fe-41ee-d1cf-08d81adfac17 X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:18:44.4084 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QRIIAS8WLxrRgX9BU3y8b1qWlGXM13kyTx3trsNUG0ZeM2rjI9RB+qKK/oSwWEHgEmz3PM7ZuKk5wK9dkGIzhA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Boris Pismenny The framework allows creating flow tables to steer incoming traffic of TCP sockets to the acceleration TIRs. This is used in downstream patches for TLS, and will be used in the future for other offloads. Signed-off-by: Boris Pismenny Signed-off-by: Tariq Toukan Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/Makefile | 2 +- .../net/ethernet/mellanox/mlx5/core/en/fs.h | 10 + .../mellanox/mlx5/core/en_accel/fs_tcp.c | 280 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/fs_tcp.h | 18 ++ .../net/ethernet/mellanox/mlx5/core/fs_core.c | 4 +- 5 files changed, 311 insertions(+), 3 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index b61e47bc16e8..8ffa1325a18f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -74,7 +74,7 @@ mlx5_core-$(CONFIG_MLX5_EN_IPSEC) += en_accel/ipsec.o en_accel/ipsec_rxtx.o \ en_accel/ipsec_stats.o mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/tls_stats.o \ - en_accel/ktls.o en_accel/ktls_tx.o + en_accel/ktls.o en_accel/ktls_tx.o en_accel/fs_tcp.o mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o \ steering/dr_matcher.o steering/dr_rule.o \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h index c633579474c3..385cbff1caf1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/fs.h @@ -123,6 +123,9 @@ enum { MLX5E_L2_FT_LEVEL, MLX5E_TTC_FT_LEVEL, MLX5E_INNER_TTC_FT_LEVEL, +#ifdef CONFIG_MLX5_EN_TLS + MLX5E_ACCEL_FS_TCP_FT_LEVEL, +#endif #ifdef CONFIG_MLX5_EN_ARFS MLX5E_ARFS_FT_LEVEL #endif @@ -216,6 +219,10 @@ static inline int mlx5e_arfs_enable(struct mlx5e_priv *priv) { return -EOPNOTSUP static inline int mlx5e_arfs_disable(struct mlx5e_priv *priv) { return -EOPNOTSUPP; } #endif +#ifdef CONFIG_MLX5_EN_TLS +struct mlx5e_accel_fs_tcp; +#endif + struct mlx5e_flow_steering { struct mlx5_flow_namespace *ns; #ifdef CONFIG_MLX5_EN_RXNFC @@ -229,6 +236,9 @@ struct mlx5e_flow_steering { #ifdef CONFIG_MLX5_EN_ARFS struct mlx5e_arfs_tables arfs; #endif +#ifdef CONFIG_MLX5_EN_TLS + struct mlx5e_accel_fs_tcp *accel_tcp; +#endif }; struct ttc_params { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c new file mode 100644 index 000000000000..a0e9082e15b0 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.c @@ -0,0 +1,280 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ + +#include +#include "en_accel/fs_tcp.h" +#include "fs_core.h" + +enum accel_fs_tcp_type { + ACCEL_FS_IPV4_TCP, + ACCEL_FS_IPV6_TCP, + ACCEL_FS_TCP_NUM_TYPES, +}; + +struct mlx5e_accel_fs_tcp { + struct mlx5e_flow_table tables[ACCEL_FS_TCP_NUM_TYPES]; + struct mlx5_flow_handle *default_rules[ACCEL_FS_TCP_NUM_TYPES]; +}; + +static enum mlx5e_traffic_types fs_accel2tt(enum accel_fs_tcp_type i) +{ + switch (i) { + case ACCEL_FS_IPV4_TCP: + return MLX5E_TT_IPV4_TCP; + default: /* ACCEL_FS_IPV6_TCP */ + return MLX5E_TT_IPV6_TCP; + } +} + +static int accel_fs_tcp_add_default_rule(struct mlx5e_priv *priv, + enum accel_fs_tcp_type type) +{ + struct mlx5e_flow_table *accel_fs_t; + struct mlx5_flow_destination dest; + struct mlx5e_accel_fs_tcp *fs_tcp; + MLX5_DECLARE_FLOW_ACT(flow_act); + struct mlx5_flow_handle *rule; + int err = 0; + + fs_tcp = priv->fs.accel_tcp; + accel_fs_t = &fs_tcp->tables[type]; + + dest = mlx5e_ttc_get_default_dest(priv, fs_accel2tt(type)); + rule = mlx5_add_flow_rules(accel_fs_t->t, NULL, &flow_act, &dest, 1); + if (IS_ERR(rule)) { + err = PTR_ERR(rule); + netdev_err(priv->netdev, + "%s: add default rule failed, accel_fs type=%d, err %d\n", + __func__, type, err); + return err; + } + + fs_tcp->default_rules[type] = rule; + return 0; +} + +#define MLX5E_ACCEL_FS_TCP_NUM_GROUPS (2) +#define MLX5E_ACCEL_FS_TCP_GROUP1_SIZE (BIT(16) - 1) +#define MLX5E_ACCEL_FS_TCP_GROUP2_SIZE (BIT(0)) +#define MLX5E_ACCEL_FS_TCP_TABLE_SIZE (MLX5E_ACCEL_FS_TCP_GROUP1_SIZE +\ + MLX5E_ACCEL_FS_TCP_GROUP2_SIZE) +static int accel_fs_tcp_create_groups(struct mlx5e_flow_table *ft, + enum accel_fs_tcp_type type) +{ + int inlen = MLX5_ST_SZ_BYTES(create_flow_group_in); + void *outer_headers_c; + int ix = 0; + u32 *in; + int err; + u8 *mc; + + ft->g = kcalloc(MLX5E_ACCEL_FS_TCP_NUM_GROUPS, sizeof(*ft->g), GFP_KERNEL); + in = kvzalloc(inlen, GFP_KERNEL); + if (!in || !ft->g) { + kvfree(ft->g); + kvfree(in); + return -ENOMEM; + } + + mc = MLX5_ADDR_OF(create_flow_group_in, in, match_criteria); + outer_headers_c = MLX5_ADDR_OF(fte_match_param, mc, outer_headers); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, ip_protocol); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, ip_version); + + switch (type) { + case ACCEL_FS_IPV4_TCP: + case ACCEL_FS_IPV6_TCP: + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, tcp_dport); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, tcp_sport); + break; + default: + err = -EINVAL; + goto out; + } + + switch (type) { + case ACCEL_FS_IPV4_TCP: + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, + src_ipv4_src_ipv6.ipv4_layout.ipv4); + MLX5_SET_TO_ONES(fte_match_set_lyr_2_4, outer_headers_c, + dst_ipv4_dst_ipv6.ipv4_layout.ipv4); + break; + case ACCEL_FS_IPV6_TCP: + memset(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_c, + src_ipv4_src_ipv6.ipv6_layout.ipv6), + 0xff, 16); + memset(MLX5_ADDR_OF(fte_match_set_lyr_2_4, outer_headers_c, + dst_ipv4_dst_ipv6.ipv6_layout.ipv6), + 0xff, 16); + break; + default: + err = -EINVAL; + goto out; + } + + MLX5_SET_CFG(in, match_criteria_enable, MLX5_MATCH_OUTER_HEADERS); + MLX5_SET_CFG(in, start_flow_index, ix); + ix += MLX5E_ACCEL_FS_TCP_GROUP1_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + /* Default Flow Group */ + memset(in, 0, inlen); + MLX5_SET_CFG(in, start_flow_index, ix); + ix += MLX5E_ACCEL_FS_TCP_GROUP2_SIZE; + MLX5_SET_CFG(in, end_flow_index, ix - 1); + ft->g[ft->num_groups] = mlx5_create_flow_group(ft->t, in); + if (IS_ERR(ft->g[ft->num_groups])) + goto err; + ft->num_groups++; + + kvfree(in); + return 0; + +err: + err = PTR_ERR(ft->g[ft->num_groups]); + ft->g[ft->num_groups] = NULL; +out: + kvfree(in); + + return err; +} + +static int accel_fs_tcp_create_table(struct mlx5e_priv *priv, enum accel_fs_tcp_type type) +{ + struct mlx5e_flow_table *ft = &priv->fs.accel_tcp->tables[type]; + struct mlx5_flow_table_attr ft_attr = {}; + int err; + + ft->num_groups = 0; + + ft_attr.max_fte = MLX5E_ACCEL_FS_TCP_TABLE_SIZE; + ft_attr.level = MLX5E_ACCEL_FS_TCP_FT_LEVEL; + ft_attr.prio = MLX5E_NIC_PRIO; + + ft->t = mlx5_create_flow_table(priv->fs.ns, &ft_attr); + if (IS_ERR(ft->t)) { + err = PTR_ERR(ft->t); + ft->t = NULL; + return err; + } + + netdev_dbg(priv->netdev, "Created fs accel table id %u level %u\n", + ft->t->id, ft->t->level); + + err = accel_fs_tcp_create_groups(ft, type); + if (err) + goto err; + + err = accel_fs_tcp_add_default_rule(priv, type); + if (err) + goto err; + + return 0; +err: + mlx5e_destroy_flow_table(ft); + return err; +} + +static int accel_fs_tcp_disable(struct mlx5e_priv *priv) +{ + int err, i; + + for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) { + /* Modify ttc rules destination to point back to the indir TIRs */ + err = mlx5e_ttc_fwd_default_dest(priv, fs_accel2tt(i)); + if (err) { + netdev_err(priv->netdev, + "%s: modify ttc[%d] default destination failed, err(%d)\n", + __func__, fs_accel2tt(i), err); + return err; + } + } + + return 0; +} + +static int accel_fs_tcp_enable(struct mlx5e_priv *priv) +{ + struct mlx5_flow_destination dest = {}; + int err, i; + + dest.type = MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE; + for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) { + dest.ft = priv->fs.accel_tcp->tables[i].t; + + /* Modify ttc rules destination to point on the accel_fs FTs */ + err = mlx5e_ttc_fwd_dest(priv, fs_accel2tt(i), &dest); + if (err) { + netdev_err(priv->netdev, + "%s: modify ttc[%d] destination to accel failed, err(%d)\n", + __func__, fs_accel2tt(i), err); + return err; + } + } + return 0; +} + +static void accel_fs_tcp_destroy_table(struct mlx5e_priv *priv, int i) +{ + struct mlx5e_accel_fs_tcp *fs_tcp; + + fs_tcp = priv->fs.accel_tcp; + if (IS_ERR_OR_NULL(fs_tcp->tables[i].t)) + return; + + mlx5_del_flow_rules(fs_tcp->default_rules[i]); + mlx5e_destroy_flow_table(&fs_tcp->tables[i]); + fs_tcp->tables[i].t = NULL; +} + +void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv) +{ + int i; + + if (!priv->fs.accel_tcp) + return; + + accel_fs_tcp_disable(priv); + + for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) + accel_fs_tcp_destroy_table(priv, i); + + kfree(priv->fs.accel_tcp); + priv->fs.accel_tcp = NULL; +} + +int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv) +{ + int i, err; + + if (!MLX5_CAP_FLOWTABLE_NIC_RX(priv->mdev, ft_field_support.outer_ip_version)) + return -EOPNOTSUPP; + + priv->fs.accel_tcp = kzalloc(sizeof(*priv->fs.accel_tcp), GFP_KERNEL); + if (!priv->fs.accel_tcp) + return -ENOMEM; + + for (i = 0; i < ACCEL_FS_TCP_NUM_TYPES; i++) { + err = accel_fs_tcp_create_table(priv, i); + if (err) + goto err_destroy_tables; + } + + err = accel_fs_tcp_enable(priv); + if (err) + goto err_destroy_tables; + + return 0; + +err_destroy_tables: + while (--i >= 0) + accel_fs_tcp_destroy_table(priv, i); + + kfree(priv->fs.accel_tcp); + priv->fs.accel_tcp = NULL; + return err; +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h new file mode 100644 index 000000000000..0df53473550a --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/fs_tcp.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */ +/* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved. */ + +#ifndef __MLX5E_ACCEL_FS_TCP_H__ +#define __MLX5E_ACCEL_FS_TCP_H__ + +#include "en.h" + +#ifdef CONFIG_MLX5_EN_TLS +int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv); +void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv); +#else +static inline int mlx5e_accel_fs_tcp_create(struct mlx5e_priv *priv) { return 0; } +static inline void mlx5e_accel_fs_tcp_destroy(struct mlx5e_priv *priv) {} +#endif + +#endif /* __MLX5E_ACCEL_FS_TCP_H__ */ + diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index e47a66983935..785b2960d6b5 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -105,8 +105,8 @@ #define ETHTOOL_PRIO_NUM_LEVELS 1 #define ETHTOOL_NUM_PRIOS 11 #define ETHTOOL_MIN_LEVEL (KERNEL_MIN_LEVEL + ETHTOOL_NUM_PRIOS) -/* Vlan, mac, ttc, inner ttc, aRFS */ -#define KERNEL_NIC_PRIO_NUM_LEVELS 5 +/* Vlan, mac, ttc, inner ttc, {aRFS/accel} */ +#define KERNEL_NIC_PRIO_NUM_LEVELS 6 #define KERNEL_NIC_NUM_PRIOS 1 /* One more level for tc */ #define KERNEL_MIN_LEVEL (KERNEL_NIC_PRIO_NUM_LEVELS + 1) From patchwork Sat Jun 27 21:17:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216972 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2385CC433DF for ; Sat, 27 Jun 2020 21:19:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id EA17B206D7 for ; Sat, 27 Jun 2020 21:19:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="YCgPMVTy" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726721AbgF0VTH (ORCPT ); Sat, 27 Jun 2020 17:19:07 -0400 Received: from mail-eopbgr60064.outbound.protection.outlook.com ([40.107.6.64]:8866 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726601AbgF0VTG (ORCPT ); Sat, 27 Jun 2020 17:19:06 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eg9wOoBqfh3xFD1eSLG4qMV2BcDWrhozOghpXPuz9QRZEO2H0oQN9b2+jZJmOTC1DKt7f8K9JkiM5t/pcLz2IUHehu+NKsx7yfUrI0e02fpvJP15kB/x2GW008AtSXdvcOjSU9vPyY3N/24bqEzKP/tlzUed+qOJcDVqd48L/ZeBX6DbzV1XqygIFpjm+eByRGsjz1JY2qTTqNLLHsS0s5GrXC2cLiRvx0taVrUgooD6gubYTsqCCXW/uHNF6nXFq7b4uG0UL4Se2qiMqyoDkAKJ3wwBi9j347bePRvZN/QZFguYP2dYrSR9YwaebMFaq7rh7qbiRA1P+r3qdk+tPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rB1EIxhsPSpBrA3cdZ6wzJBbukNCV4D08mBoezhIC6I=; b=TwVftglx3fXyW5Z6rIC3Aj3JDgWCdWP0qOsHU1UWp+wmvAH/IEieUBUFWv74KITD9WhBGkxTduy/1HISgxgt7Zz+Trmt6oejujqWKZ/7oX4bB3f58Q9OjEm/vs1bCHT9r07IRlIIVq9wMEQBJV5qGQlD6LeWao2VQhBxdsjqhPZ74UqBCwTGswzi8Pyt8WF37AktywUfzjnSrbR37e0oJGK0hVLysoMcwdSv3FKLlOU58MFx4re/4MD/cbvJUGaAJDTlSvBzjc7qNyt3s2rJDpsREpk4pu/7xLp4DXq0JHJiawAgOLjIhrfwsCF1zKNH+35OKgY8HKkgyMfdE4M7uQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rB1EIxhsPSpBrA3cdZ6wzJBbukNCV4D08mBoezhIC6I=; b=YCgPMVTyoi47RCCF+IpPUZx276QOEyXutNmn8N7uWD+hEb2hj2WfD4D4ZWKvg8hMAYUPnaFMpMimW2MqKts3dCvytFpxLkiXJeOM8zr+4V5aVkTBs2/7rrax5FMv22sa1CmRKL3fcnCdQX24It1u43oVZhUlturguFMqGpiIGXM= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:18:51 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:18:51 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Tariq Toukan , Maxim Mikityanskiy , Saeed Mahameed Subject: [net-next 07/15] net/mlx5e: kTLS, Use kernel API to extract private offload context Date: Sat, 27 Jun 2020 14:17:19 -0700 Message-Id: <20200627211727.259569-8-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:18:49 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 9b234d36-848e-494e-a8a8-08d81adfb05c X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:962; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: OT6HEDUqWpzywGRQVFWCfWTPw3sP6/CVmuyt4aXSM6hY9E3JGSmrXAyvlP8fb5+q+yYsMyV9dMz2aej5i6AxjuxHiK8QsDCIby0TZg3Uv8p8BIRj/0fqyYflAPn8Af0pnRkRQVlJYLrXsq0jreQC651PRI3vxZRHYZ+83HTZkQ6MOOUvQL9W1dBFHELLS2AZeQbaId8Nrfz2lYtYedvyZ6sz51uvP9gOmSlhMrmHqyAA6BoGOrOfcpUA4HvmADca1j9J5hMScf2Aw5EYnECVU5e80Ae7y/C6AQBsfXSLMTJUly6GnFd1r/cenf8adPyLc7JnuoDHqJye0xKxyV0FsOHeib1k1SUSCECele7zUS+fAPN/NKrXjk3zizWwRnIV X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: wqLtymf0vHgwH/ek7Aik5ezm67oN55BCYkKNv52irDfMDL9OSdJ9B6nMMTya3FKwo3MW69dLtTiOXeBD9knhhifJ3UFRnkYf6rnvD7In+fwm8ky4lTBjbmx4gOR3eSSr6jKw6mU+oW1ew2OnW2Kb7B3NDs23mEgsBumjxWN7iX5BsXnKlxPJtSXQ1vp+Lq6HXFykMfMB/6th5Pb2QtPU5uEdAovWFtjYFe+px1/MeqGYztCDErhaEuyJZwib3jlDx3B66dYMFK+nyz8Kc5Wb6ro1XzQfy2877KqWpAFX64ZaDie96didurq2e7k/8UWDUUJqxCjJcA9aZPKtGcbDeOBSEGpOiOA6zM1GJTbL4LpEAeTtsrk6YGukns0/YbRXonsHUDF3BkE4yYagBnmCUfamNpgjEPQdMj8Ty46WXwXP0vy9lKgH4eVb0BwCPNmHwWJSPTlbIlJhZvu2KIN/TCoFW0uvx0rvria6PQ2sgU8= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9b234d36-848e-494e-a8a8-08d81adfb05c X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:18:51.3875 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PPhM46jugEWqMUMpwdsFfcyMAJUVdsIPiSotiyCgODbZuFHMcpesucuR6yPv20hkwvBS7exfMHOuiqYc7CF5Gw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tariq Toukan Modify the implementation of the private kTLS TX HW offload context getter and setter, so it uses the kernel API functions, instead of a local shadow structure. A single BUILD_BUG_ON check is sufficient, remove the duplicate. Signed-off-by: Tariq Toukan Reviewed-by: Maxim Mikityanskiy Signed-off-by: Saeed Mahameed --- .../mellanox/mlx5/core/en_accel/ktls_tx.c | 28 ++++++------------- 1 file changed, 9 insertions(+), 19 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c index 349e29214b92..5a980f93c326 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c @@ -56,37 +56,26 @@ struct mlx5e_ktls_offload_context_tx { bool ctx_post_pending; }; -struct mlx5e_ktls_offload_context_tx_shadow { - struct tls_offload_context_tx tx_ctx; - struct mlx5e_ktls_offload_context_tx *priv_tx; -}; - static void mlx5e_set_ktls_tx_priv_ctx(struct tls_context *tls_ctx, struct mlx5e_ktls_offload_context_tx *priv_tx) { - struct tls_offload_context_tx *tx_ctx = tls_offload_ctx_tx(tls_ctx); - struct mlx5e_ktls_offload_context_tx_shadow *shadow; + struct mlx5e_ktls_offload_context_tx **ctx = + __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX); - BUILD_BUG_ON(sizeof(*shadow) > TLS_OFFLOAD_CONTEXT_SIZE_TX); + BUILD_BUG_ON(sizeof(struct mlx5e_ktls_offload_context_tx *) > + TLS_OFFLOAD_CONTEXT_SIZE_TX); - shadow = (struct mlx5e_ktls_offload_context_tx_shadow *)tx_ctx; - - shadow->priv_tx = priv_tx; - priv_tx->tx_ctx = tx_ctx; + *ctx = priv_tx; } static struct mlx5e_ktls_offload_context_tx * mlx5e_get_ktls_tx_priv_ctx(struct tls_context *tls_ctx) { - struct tls_offload_context_tx *tx_ctx = tls_offload_ctx_tx(tls_ctx); - struct mlx5e_ktls_offload_context_tx_shadow *shadow; - - BUILD_BUG_ON(sizeof(*shadow) > TLS_OFFLOAD_CONTEXT_SIZE_TX); - - shadow = (struct mlx5e_ktls_offload_context_tx_shadow *)tx_ctx; + struct mlx5e_ktls_offload_context_tx **ctx = + __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_TX); - return shadow->priv_tx; + return *ctx; } int mlx5e_ktls_add_tx(struct net_device *netdev, struct sock *sk, @@ -113,6 +102,7 @@ int mlx5e_ktls_add_tx(struct net_device *netdev, struct sock *sk, priv_tx->expected_seq = start_offload_tcp_sn; priv_tx->crypto_info = *(struct tls12_crypto_info_aes_gcm_128 *)crypto_info; + priv_tx->tx_ctx = tls_offload_ctx_tx(tls_ctx); mlx5e_set_ktls_tx_priv_ctx(tls_ctx, priv_tx); From patchwork Sat Jun 27 21:17:20 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6B336C433DF for ; Sat, 27 Jun 2020 21:19:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 28A31206D7 for ; Sat, 27 Jun 2020 21:19:21 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="eRNrMvrl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726806AbgF0VTU (ORCPT ); Sat, 27 Jun 2020 17:19:20 -0400 Received: from mail-eopbgr60064.outbound.protection.outlook.com ([40.107.6.64]:8866 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725823AbgF0VTP (ORCPT ); Sat, 27 Jun 2020 17:19:15 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=hNNk628PR9z3yAU8n5RcbfX0USPs3I9Eow4RcolMiOgUMhB7ys50R5DN2K4jLnq4NrJZXzxxW3RJ4X/bpcb/dtJsWU8n7OUxi0+xeHZ/kRcaVa7vZwcSN+Wqs/BspzpnmN9wlNLBwYQSQO5FBQmJGHDN0eGyiuxEiF1CQ7+xFAQV5qgQs6QpsgTwNgofACw8ssuu0EDIln0YngPin4AhEQVzfwIX9KCHJNPxD2udhxLOTZPzw7pabFBq1lUNa7pHjm22ieRRWP/CMDx5XD7+LliYqZIZL0+IvGiAmHZkQD8dp5lQwfwc0BJdToUOaFNz99HgF1s2tDXrrtretcblPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ksGnwQzB7AtUN+czh2bnPhfWOLjn21UA5oXIu1ZHAcU=; b=IIMpYA9p/gPiqZL4hyLjGQqOYHaF/20W9QpuK98x/WOZ82dqmKRx1db45caJ5LJ+bH1ptGJM60Hlgr9Thi76BBCFmAXQLCxtVk/BolR+RFtyQPtOlVwkBp3yxhTxLjeDHdFzfIUT1PoB6WTbpyQS6gGL/CvyTIIVikIAk5eHMoRf85B7KtTnUNSFyQInPugWtzIOiNmFoxq6b6U1UNMP9tyHKbTtCMFAz8oPL0jdudPaCII6oZsBR+NZuUF9l7u5lzJb3eDGS6NqXqvzYZJn5cDWv+I0HHT3ibCKIinVY8Z1ONQvHOA+Z7OCFVyNwTOlcTkD9QMPl8/bj3LBDpfvDA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ksGnwQzB7AtUN+czh2bnPhfWOLjn21UA5oXIu1ZHAcU=; b=eRNrMvrlvZsqQKMYkCxzRAolF8kQxt2dnSoPQseJt6YV2kJEb76fpi8VZ8stiMykXsWMKRflc6xUQqkQblZBRCg5KGqwr1neor6oyMirKaOVs1LY40PgvBtiOpYUw0cRzVNQBMl9/QHUx50kEm/N5xh/B1ws7QVAqFZ5H+2vNWI= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:18:53 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:18:53 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Tariq Toukan , Saeed Mahameed Subject: [net-next 08/15] net/mlx5e: kTLS, Add kTLS RX HW offload support Date: Sat, 27 Jun 2020 14:17:20 -0700 Message-Id: <20200627211727.259569-9-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:18:51 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: f13677e9-fa10-482e-6914-08d81adfb190 X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2582; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NXTF7X2eHVBFyUtC+vw0Jm536KtzbI95G2NxXnUmmqea+H3FXp97jXGxKVXQFVWmAveHZ1+Nwc0j3cQU6wpMfn+Um8aokQ67g/a1ETWPXUVLKiXJcCu5UDXnub5hMLknI8HzUZ9h2DpWu4LtDalZQT0zkDgrFwcVGvQ47GIbFyH4t6z/XpybR6ZSzUlRllYUeg7TzI4f3u1DL4QEk80gqn9IchfICldSPgW+sp0qgksAQLDHWBAid+qqSrnnNE4VqrVFfCctKbhQu6KkUU2chFWWy7bbpQgkdanTGgBcZmDDdFFvlTOPWAp8uxGRTatpOWQrqscUrj7YiXNAotAeQjb2rFw6MPNuuLHAwBpVVppcmPoA4DiE5UyqU6wTH/GBp1TnZLWFbHkZ/W+jQTewaO+jkTvb4Cd/RtSh0Xr15hE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(30864003)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002)(309714004); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: ygAyvYbIGd4vLAynaQZd2Xu2OB8reTpHgjlQRAFIcNdwCcF3cF1zh54ppLdENaOtA+kCWpLFnQ8QGnqUfu9ub4GkSWkSyFAA0LxUedtxQjn7dDakD8BDUtlcD79d2N0azCyMGRnumNt5YrQwGaVarJyEVtFj0RBe+LbEcH9DYO48/z+yYsVlrFgYGJfzh79B+TVuKKdJXEeBebvzkzugHnnj9VvgZh0D54uhkS+1/oioXHVWJAghTflybYvp2PPqRdJe4g19a8Ip6qCQF4PTomzE90vgS1WEEOphqh1q6B/hLcifiTPCe8zDGP8Gfch7GycPjtzxcRS0vqzUO6uttLuipD1aNK7eNTTYVTwCgyqenNLsF7an5O5tz9xfWYR5HETdVdEYXfHhuBTc2eNjPnbHN4J0cmAV/7Ru38TFuJt+LbkiEEie3RWln/6yTFyBkSUdHeJHrrAoqszyRQtm1CHPJfScDPxOhewnzFj6+dM= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: f13677e9-fa10-482e-6914-08d81adfb190 X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:18:53.5203 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ku6IYlubGlgMoJbwm7DZ7o2xPWDrSYOl73hmYR+SkFK9r+Z3ola5YJ7HLSmvslG5R7oJU7W86ADzKfYvGg7HzQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tariq Toukan Implement driver support for the kTLS RX HW offload feature. Resync support is added in a downstream patch. New offload contexts post their static/progress params WQEs over the per-channel async ICOSQ, protected under a spin-lock. The Channel/RQ is selected according to the socket's rxq index. Feature is OFF by default. Can be turned on by: $ ethtool -K tls-hw-rx-offload on A new TLS-RX workqueue is used to allow asynchronous addition of steering rules, out of the NAPI context. It will be also used in a downstream patch in the resync procedure. Signed-off-by: Tariq Toukan Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/Kconfig | 1 + .../net/ethernet/mellanox/mlx5/core/Makefile | 2 +- .../ethernet/mellanox/mlx5/core/accel/tls.h | 19 +- .../net/ethernet/mellanox/mlx5/core/en/txrx.h | 11 + .../mellanox/mlx5/core/en_accel/en_accel.h | 20 ++ .../mellanox/mlx5/core/en_accel/ktls.c | 66 +++- .../mellanox/mlx5/core/en_accel/ktls.h | 19 +- .../mellanox/mlx5/core/en_accel/ktls_rx.c | 311 ++++++++++++++++++ .../mellanox/mlx5/core/en_accel/ktls_tx.c | 2 +- .../mellanox/mlx5/core/en_accel/ktls_txrx.c | 20 +- .../mellanox/mlx5/core/en_accel/ktls_txrx.h | 4 + .../mellanox/mlx5/core/en_accel/ktls_utils.h | 4 + .../mellanox/mlx5/core/en_accel/tls.c | 12 +- .../mellanox/mlx5/core/en_accel/tls.h | 1 + .../mellanox/mlx5/core/en_accel/tls_rxtx.c | 20 +- .../mellanox/mlx5/core/en_accel/tls_rxtx.h | 4 +- .../net/ethernet/mellanox/mlx5/core/en_main.c | 9 + .../net/ethernet/mellanox/mlx5/core/en_rx.c | 33 +- drivers/net/ethernet/mellanox/mlx5/core/fw.c | 3 +- 19 files changed, 529 insertions(+), 32 deletions(-) create mode 100644 drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig index 4dfdbb82ea9d..76b39659c39b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Kconfig +++ b/drivers/net/ethernet/mellanox/mlx5/core/Kconfig @@ -173,6 +173,7 @@ config MLX5_TLS config MLX5_EN_TLS bool "TLS cryptography-offload accelaration" depends on MLX5_CORE_EN + depends on XPS depends on MLX5_FPGA_TLS || MLX5_TLS default y help diff --git a/drivers/net/ethernet/mellanox/mlx5/core/Makefile b/drivers/net/ethernet/mellanox/mlx5/core/Makefile index 70ad24fff2e2..1e7c7f10db6e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/Makefile +++ b/drivers/net/ethernet/mellanox/mlx5/core/Makefile @@ -75,7 +75,7 @@ mlx5_core-$(CONFIG_MLX5_EN_IPSEC) += en_accel/ipsec.o en_accel/ipsec_rxtx.o \ mlx5_core-$(CONFIG_MLX5_EN_TLS) += en_accel/tls.o en_accel/tls_rxtx.o en_accel/tls_stats.o \ en_accel/fs_tcp.o en_accel/ktls.o en_accel/ktls_txrx.o \ - en_accel/ktls_tx.o + en_accel/ktls_tx.o en_accel/ktls_rx.o mlx5_core-$(CONFIG_MLX5_SW_STEERING) += steering/dr_domain.o steering/dr_table.o \ steering/dr_matcher.o steering/dr_rule.o \ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.h b/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.h index aefea467f7b3..fd874f0c380a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/accel/tls.h @@ -43,9 +43,20 @@ int mlx5_ktls_create_key(struct mlx5_core_dev *mdev, u32 *p_key_id); void mlx5_ktls_destroy_key(struct mlx5_core_dev *mdev, u32 key_id); +static inline bool mlx5_accel_is_ktls_tx(struct mlx5_core_dev *mdev) +{ + return MLX5_CAP_GEN(mdev, tls_tx); +} + +static inline bool mlx5_accel_is_ktls_rx(struct mlx5_core_dev *mdev) +{ + return MLX5_CAP_GEN(mdev, tls_rx); +} + static inline bool mlx5_accel_is_ktls_device(struct mlx5_core_dev *mdev) { - if (!MLX5_CAP_GEN(mdev, tls_tx)) + if (!mlx5_accel_is_ktls_tx(mdev) && + !mlx5_accel_is_ktls_rx(mdev)) return false; if (!MLX5_CAP_GEN(mdev, log_max_dek)) @@ -67,6 +78,12 @@ static inline bool mlx5e_ktls_type_check(struct mlx5_core_dev *mdev, return false; } #else +static inline bool mlx5_accel_is_ktls_tx(struct mlx5_core_dev *mdev) +{ return false; } + +static inline bool mlx5_accel_is_ktls_rx(struct mlx5_core_dev *mdev) +{ return false; } + static inline int mlx5_ktls_create_key(struct mlx5_core_dev *mdev, struct tls_crypto_info *crypto_info, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h index 31cac239563d..7f55bd3229f1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h @@ -11,6 +11,10 @@ enum mlx5e_icosq_wqe_type { MLX5E_ICOSQ_WQE_NOP, MLX5E_ICOSQ_WQE_UMR_RX, +#ifdef CONFIG_MLX5_EN_TLS + MLX5E_ICOSQ_WQE_UMR_TLS, + MLX5E_ICOSQ_WQE_SET_PSV_TLS, +#endif }; static inline bool @@ -114,9 +118,16 @@ struct mlx5e_icosq_wqe_info { struct { struct mlx5e_rq *rq; } umr; +#ifdef CONFIG_MLX5_EN_TLS + struct { + struct mlx5e_ktls_offload_context_rx *priv_rx; + } tls_set_params; +#endif }; }; +void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq); + static inline u16 mlx5e_icosq_get_next_pi(struct mlx5e_icosq *sq, u16 size) { struct mlx5_wq_cyc *wq = &sq->wq; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h index fac145dcf2ce..7b6abea850d4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h @@ -37,6 +37,7 @@ #include #include #include "en_accel/ipsec_rxtx.h" +#include "en_accel/tls.h" #include "en_accel/tls_rxtx.h" #include "en.h" #include "en/txrx.h" @@ -147,4 +148,23 @@ static inline bool mlx5e_accel_tx_finish(struct mlx5e_priv *priv, return true; } +static inline int mlx5e_accel_sk_get_rxq(struct sock *sk) +{ + int rxq = sk_rx_queue_get(sk); + + if (unlikely(rxq == -1)) + rxq = 0; + + return rxq; +} + +static inline int mlx5e_accel_init_rx(struct mlx5e_priv *priv) +{ + return mlx5e_ktls_init_rx(priv); +} + +static inline void mlx5e_accel_cleanup_rx(struct mlx5e_priv *priv) +{ + mlx5e_ktls_cleanup_rx(priv); +} #endif /* __MLX5E_EN_ACCEL_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c index 8970ea68d005..d4ef016ab444 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.c @@ -4,6 +4,7 @@ #include "en.h" #include "en_accel/ktls.h" #include "en_accel/ktls_utils.h" +#include "en_accel/fs_tcp.h" static int mlx5e_ktls_add(struct net_device *netdev, struct sock *sk, enum tls_offload_ctx_dir direction, @@ -14,13 +15,13 @@ static int mlx5e_ktls_add(struct net_device *netdev, struct sock *sk, struct mlx5_core_dev *mdev = priv->mdev; int err; - if (WARN_ON(direction != TLS_OFFLOAD_CTX_DIR_TX)) - return -EINVAL; - if (WARN_ON(!mlx5e_ktls_type_check(mdev, crypto_info))) return -EOPNOTSUPP; - err = mlx5e_ktls_add_tx(netdev, sk, crypto_info, start_offload_tcp_sn); + if (direction == TLS_OFFLOAD_CTX_DIR_TX) + err = mlx5e_ktls_add_tx(netdev, sk, crypto_info, start_offload_tcp_sn); + else + err = mlx5e_ktls_add_rx(netdev, sk, crypto_info, start_offload_tcp_sn); return err; } @@ -29,26 +30,71 @@ static void mlx5e_ktls_del(struct net_device *netdev, struct tls_context *tls_ctx, enum tls_offload_ctx_dir direction) { - if (direction != TLS_OFFLOAD_CTX_DIR_TX) - return; + if (direction == TLS_OFFLOAD_CTX_DIR_TX) + mlx5e_ktls_del_tx(netdev, tls_ctx); + else + mlx5e_ktls_del_rx(netdev, tls_ctx); +} - mlx5e_ktls_del_tx(netdev, tls_ctx); +static int mlx5e_ktls_resync(struct net_device *netdev, + struct sock *sk, u32 seq, u8 *rcd_sn, + enum tls_offload_ctx_dir direction) +{ + return -EOPNOTSUPP; } static const struct tlsdev_ops mlx5e_ktls_ops = { .tls_dev_add = mlx5e_ktls_add, .tls_dev_del = mlx5e_ktls_del, + .tls_dev_resync = mlx5e_ktls_resync, }; void mlx5e_ktls_build_netdev(struct mlx5e_priv *priv) { struct net_device *netdev = priv->netdev; + struct mlx5_core_dev *mdev = priv->mdev; - if (!mlx5_accel_is_ktls_device(priv->mdev)) + if (!mlx5_accel_is_ktls_device(mdev)) return; - netdev->hw_features |= NETIF_F_HW_TLS_TX; - netdev->features |= NETIF_F_HW_TLS_TX; + if (mlx5_accel_is_ktls_tx(mdev)) { + netdev->hw_features |= NETIF_F_HW_TLS_TX; + netdev->features |= NETIF_F_HW_TLS_TX; + } + + if (mlx5_accel_is_ktls_rx(mdev)) + netdev->hw_features |= NETIF_F_HW_TLS_RX; netdev->tlsdev_ops = &mlx5e_ktls_ops; } + +int mlx5e_ktls_set_feature_rx(struct net_device *netdev, bool enable) +{ + struct mlx5e_priv *priv = netdev_priv(netdev); + int err = 0; + + mutex_lock(&priv->state_lock); + if (enable) + err = mlx5e_accel_fs_tcp_create(priv); + else + mlx5e_accel_fs_tcp_destroy(priv); + mutex_unlock(&priv->state_lock); + + return err; +} + +int mlx5e_ktls_init_rx(struct mlx5e_priv *priv) +{ + int err = 0; + + if (priv->netdev->features & NETIF_F_HW_TLS_RX) + err = mlx5e_accel_fs_tcp_create(priv); + + return err; +} + +void mlx5e_ktls_cleanup_rx(struct mlx5e_priv *priv) +{ + if (priv->netdev->features & NETIF_F_HW_TLS_RX) + mlx5e_accel_fs_tcp_destroy(priv); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h index 69d736954977..baa58b62e8df 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h @@ -9,13 +9,30 @@ #ifdef CONFIG_MLX5_EN_TLS void mlx5e_ktls_build_netdev(struct mlx5e_priv *priv); - +int mlx5e_ktls_init_rx(struct mlx5e_priv *priv); +void mlx5e_ktls_cleanup_rx(struct mlx5e_priv *priv); +int mlx5e_ktls_set_feature_rx(struct net_device *netdev, bool enable); #else static inline void mlx5e_ktls_build_netdev(struct mlx5e_priv *priv) { } +static inline int mlx5e_ktls_init_rx(struct mlx5e_priv *priv) +{ + return 0; +} + +static inline void mlx5e_ktls_cleanup_rx(struct mlx5e_priv *priv) +{ +} + +static inline int mlx5e_ktls_set_feature_rx(struct net_device *netdev, bool enable) +{ + netdev_warn(netdev, "kTLS is not supported\n"); + return -EOPNOTSUPP; +} + #endif #endif /* __MLX5E_TLS_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c new file mode 100644 index 000000000000..aae4245d0c91 --- /dev/null +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c @@ -0,0 +1,311 @@ +// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB +// Copyright (c) 2019 Mellanox Technologies. + +#include "en_accel/en_accel.h" +#include "en_accel/ktls_txrx.h" +#include "en_accel/ktls_utils.h" +#include "en_accel/fs_tcp.h" + +struct accel_rule { + struct work_struct work; + struct mlx5e_priv *priv; + struct mlx5_flow_handle *rule; +}; + +enum { + MLX5E_PRIV_RX_FLAG_DELETING, + MLX5E_NUM_PRIV_RX_FLAGS, +}; + +struct mlx5e_ktls_offload_context_rx { + struct tls12_crypto_info_aes_gcm_128 crypto_info; + struct accel_rule rule; + struct sock *sk; + struct completion add_ctx; + u32 tirn; + u32 key_id; + u32 rxq; + DECLARE_BITMAP(flags, MLX5E_NUM_PRIV_RX_FLAGS); +}; + +static int mlx5e_ktls_create_tir(struct mlx5_core_dev *mdev, u32 *tirn, u32 rqtn) +{ + int err, inlen; + void *tirc; + u32 *in; + + inlen = MLX5_ST_SZ_BYTES(create_tir_in); + in = kvzalloc(inlen, GFP_KERNEL); + if (!in) + return -ENOMEM; + + tirc = MLX5_ADDR_OF(create_tir_in, in, ctx); + + MLX5_SET(tirc, tirc, transport_domain, mdev->mlx5e_res.td.tdn); + MLX5_SET(tirc, tirc, disp_type, MLX5_TIRC_DISP_TYPE_INDIRECT); + MLX5_SET(tirc, tirc, rx_hash_fn, MLX5_RX_HASH_FN_INVERTED_XOR8); + MLX5_SET(tirc, tirc, indirect_table, rqtn); + MLX5_SET(tirc, tirc, tls_en, 1); + MLX5_SET(tirc, tirc, self_lb_block, + MLX5_TIRC_SELF_LB_BLOCK_BLOCK_UNICAST | + MLX5_TIRC_SELF_LB_BLOCK_BLOCK_MULTICAST); + + err = mlx5_core_create_tir(mdev, in, tirn); + + kvfree(in); + return err; +} + +static void accel_rule_handle_work(struct work_struct *work) +{ + struct mlx5e_ktls_offload_context_rx *priv_rx; + struct accel_rule *accel_rule; + struct mlx5_flow_handle *rule; + + accel_rule = container_of(work, struct accel_rule, work); + priv_rx = container_of(accel_rule, struct mlx5e_ktls_offload_context_rx, rule); + if (unlikely(test_bit(MLX5E_PRIV_RX_FLAG_DELETING, priv_rx->flags))) + goto out; + + rule = mlx5e_accel_fs_add_sk(accel_rule->priv, priv_rx->sk, + priv_rx->tirn, MLX5_FS_DEFAULT_FLOW_TAG); + if (!IS_ERR_OR_NULL(rule)) + accel_rule->rule = rule; +out: + complete(&priv_rx->add_ctx); +} + +static void accel_rule_init(struct accel_rule *rule, struct mlx5e_priv *priv, + struct sock *sk) +{ + INIT_WORK(&rule->work, accel_rule_handle_work); + rule->priv = priv; +} + +static void icosq_fill_wi(struct mlx5e_icosq *sq, u16 pi, + struct mlx5e_icosq_wqe_info *wi) +{ + sq->db.wqe_info[pi] = *wi; +} + +static struct mlx5_wqe_ctrl_seg * +post_static_params(struct mlx5e_icosq *sq, + struct mlx5e_ktls_offload_context_rx *priv_rx) +{ + struct mlx5e_set_tls_static_params_wqe *wqe; + struct mlx5e_icosq_wqe_info wi; + u16 pi, num_wqebbs, room; + + num_wqebbs = MLX5E_TLS_SET_STATIC_PARAMS_WQEBBS; + room = mlx5e_stop_room_for_wqe(num_wqebbs); + if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room))) + return ERR_PTR(-ENOSPC); + + pi = mlx5e_icosq_get_next_pi(sq, num_wqebbs); + wqe = MLX5E_TLS_FETCH_SET_STATIC_PARAMS_WQE(sq, pi); + mlx5e_ktls_build_static_params(wqe, sq->pc, sq->sqn, &priv_rx->crypto_info, + priv_rx->tirn, priv_rx->key_id, false, + TLS_OFFLOAD_CTX_DIR_RX); + wi = (struct mlx5e_icosq_wqe_info) { + .wqe_type = MLX5E_ICOSQ_WQE_UMR_TLS, + .num_wqebbs = num_wqebbs, + .tls_set_params.priv_rx = priv_rx, + }; + icosq_fill_wi(sq, pi, &wi); + sq->pc += num_wqebbs; + + return &wqe->ctrl; +} + +static struct mlx5_wqe_ctrl_seg * +post_progress_params(struct mlx5e_icosq *sq, + struct mlx5e_ktls_offload_context_rx *priv_rx, + u32 next_record_tcp_sn) +{ + struct mlx5e_set_tls_progress_params_wqe *wqe; + struct mlx5e_icosq_wqe_info wi; + u16 pi, num_wqebbs, room; + + num_wqebbs = MLX5E_TLS_SET_PROGRESS_PARAMS_WQEBBS; + room = mlx5e_stop_room_for_wqe(num_wqebbs); + if (unlikely(!mlx5e_wqc_has_room_for(&sq->wq, sq->cc, sq->pc, room))) + return ERR_PTR(-ENOSPC); + + pi = mlx5e_icosq_get_next_pi(sq, num_wqebbs); + wqe = MLX5E_TLS_FETCH_SET_PROGRESS_PARAMS_WQE(sq, pi); + mlx5e_ktls_build_progress_params(wqe, sq->pc, sq->sqn, priv_rx->tirn, false, + next_record_tcp_sn, + TLS_OFFLOAD_CTX_DIR_RX); + wi = (struct mlx5e_icosq_wqe_info) { + .wqe_type = MLX5E_ICOSQ_WQE_SET_PSV_TLS, + .num_wqebbs = num_wqebbs, + .tls_set_params.priv_rx = priv_rx, + }; + + icosq_fill_wi(sq, pi, &wi); + sq->pc += num_wqebbs; + + return &wqe->ctrl; +} + +static int post_rx_param_wqes(struct mlx5e_channel *c, + struct mlx5e_ktls_offload_context_rx *priv_rx, + u32 next_record_tcp_sn) +{ + struct mlx5_wqe_ctrl_seg *cseg; + struct mlx5e_icosq *sq; + int err; + + err = 0; + sq = &c->async_icosq; + spin_lock(&c->async_icosq_lock); + + cseg = post_static_params(sq, priv_rx); + if (IS_ERR(cseg)) + goto err_out; + cseg = post_progress_params(sq, priv_rx, next_record_tcp_sn); + if (IS_ERR(cseg)) + goto err_out; + + mlx5e_notify_hw(&sq->wq, sq->pc, sq->uar_map, cseg); +unlock: + spin_unlock(&c->async_icosq_lock); + + return err; + +err_out: + err = PTR_ERR(cseg); + complete(&priv_rx->add_ctx); + goto unlock; +} + +static void +mlx5e_set_ktls_rx_priv_ctx(struct tls_context *tls_ctx, + struct mlx5e_ktls_offload_context_rx *priv_rx) +{ + struct mlx5e_ktls_offload_context_rx **ctx = + __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_RX); + + BUILD_BUG_ON(sizeof(struct mlx5e_ktls_offload_context_rx *) > + TLS_OFFLOAD_CONTEXT_SIZE_RX); + + *ctx = priv_rx; +} + +static struct mlx5e_ktls_offload_context_rx * +mlx5e_get_ktls_rx_priv_ctx(struct tls_context *tls_ctx) +{ + struct mlx5e_ktls_offload_context_rx **ctx = + __tls_driver_ctx(tls_ctx, TLS_OFFLOAD_CTX_DIR_RX); + + return *ctx; +} + +void mlx5e_ktls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, + struct mlx5_cqe64 *cqe, u32 *cqe_bcnt) +{ + u8 tls_offload = get_cqe_tls_offload(cqe); + + if (likely(tls_offload == CQE_TLS_OFFLOAD_NOT_DECRYPTED)) + return; + + switch (tls_offload) { + case CQE_TLS_OFFLOAD_DECRYPTED: + skb->decrypted = 1; + break; + case CQE_TLS_OFFLOAD_RESYNC: + break; + default: /* CQE_TLS_OFFLOAD_ERROR: */ + break; + } +} + +void mlx5e_ktls_handle_ctx_completion(struct mlx5e_icosq_wqe_info *wi) +{ + struct mlx5e_ktls_offload_context_rx *priv_rx = wi->tls_set_params.priv_rx; + struct accel_rule *rule = &priv_rx->rule; + + if (unlikely(test_bit(MLX5E_PRIV_RX_FLAG_DELETING, priv_rx->flags))) { + complete(&priv_rx->add_ctx); + return; + } + queue_work(rule->priv->tls->rx_wq, &rule->work); +} + +int mlx5e_ktls_add_rx(struct net_device *netdev, struct sock *sk, + struct tls_crypto_info *crypto_info, + u32 start_offload_tcp_sn) +{ + struct mlx5e_ktls_offload_context_rx *priv_rx; + struct tls_context *tls_ctx; + struct mlx5_core_dev *mdev; + struct mlx5e_priv *priv; + int rxq, err; + u32 rqtn; + + tls_ctx = tls_get_ctx(sk); + priv = netdev_priv(netdev); + mdev = priv->mdev; + priv_rx = kzalloc(sizeof(*priv_rx), GFP_KERNEL); + if (unlikely(!priv_rx)) + return -ENOMEM; + + err = mlx5_ktls_create_key(mdev, crypto_info, &priv_rx->key_id); + if (err) + goto err_create_key; + + priv_rx->crypto_info = + *(struct tls12_crypto_info_aes_gcm_128 *)crypto_info; + priv_rx->sk = sk; + priv_rx->rxq = mlx5e_accel_sk_get_rxq(sk); + + mlx5e_set_ktls_rx_priv_ctx(tls_ctx, priv_rx); + + rxq = priv_rx->rxq; + rqtn = priv->direct_tir[rxq].rqt.rqtn; + + err = mlx5e_ktls_create_tir(mdev, &priv_rx->tirn, rqtn); + if (err) + goto err_create_tir; + + init_completion(&priv_rx->add_ctx); + accel_rule_init(&priv_rx->rule, priv, sk); + err = post_rx_param_wqes(priv->channels.c[rxq], priv_rx, start_offload_tcp_sn); + if (err) + goto err_post_wqes; + + return 0; + +err_post_wqes: + mlx5_core_destroy_tir(mdev, priv_rx->tirn); +err_create_tir: + mlx5_ktls_destroy_key(mdev, priv_rx->key_id); +err_create_key: + kfree(priv_rx); + return err; +} + +void mlx5e_ktls_del_rx(struct net_device *netdev, struct tls_context *tls_ctx) +{ + struct mlx5e_ktls_offload_context_rx *priv_rx; + struct mlx5_core_dev *mdev; + struct mlx5e_priv *priv; + + priv = netdev_priv(netdev); + mdev = priv->mdev; + + priv_rx = mlx5e_get_ktls_rx_priv_ctx(tls_ctx); + set_bit(MLX5E_PRIV_RX_FLAG_DELETING, priv_rx->flags); + if (!cancel_work_sync(&priv_rx->rule.work)) + /* completion is needed, as the priv_rx in the add flow + * is maintained on the wqe info (wi), not on the socket. + */ + wait_for_completion(&priv_rx->add_ctx); + + if (priv_rx->rule.rule) + mlx5e_accel_fs_del_sk(priv_rx->rule.rule); + + mlx5_core_destroy_tir(mdev, priv_rx->tirn); + mlx5_ktls_destroy_key(mdev, priv_rx->key_id); + kfree(priv_rx); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c index 5a980f93c326..9c34ffa55b32 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c @@ -188,7 +188,7 @@ post_progress_params(struct mlx5e_txqsq *sq, num_wqebbs = MLX5E_TLS_SET_PROGRESS_PARAMS_WQEBBS; pi = mlx5e_txqsq_get_next_pi(sq, num_wqebbs); wqe = MLX5E_TLS_FETCH_SET_PROGRESS_PARAMS_WQE(sq, pi); - mlx5e_ktls_build_progress_params(wqe, sq->pc, sq->sqn, priv_tx->tisn, fence, + mlx5e_ktls_build_progress_params(wqe, sq->pc, sq->sqn, priv_tx->tisn, fence, 0, TLS_OFFLOAD_CTX_DIR_TX); tx_fill_wi(sq, pi, num_wqebbs, 0, NULL); sq->pc += num_wqebbs; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c index edf404eaa275..c1f1ad32ca4c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.c @@ -59,11 +59,13 @@ mlx5e_ktls_build_static_params(struct mlx5e_set_tls_static_params_wqe *wqe, { struct mlx5_wqe_umr_ctrl_seg *ucseg = &wqe->uctrl; struct mlx5_wqe_ctrl_seg *cseg = &wqe->ctrl; + u8 opmod = direction == TLS_OFFLOAD_CTX_DIR_TX ? + MLX5_OPC_MOD_TLS_TIS_STATIC_PARAMS : + MLX5_OPC_MOD_TLS_TIR_STATIC_PARAMS; #define STATIC_PARAMS_DS_CNT DIV_ROUND_UP(sizeof(*wqe), MLX5_SEND_WQE_DS) - cseg->opmod_idx_opcode = cpu_to_be32((pc << 8) | MLX5_OPCODE_UMR | - (MLX5_OPC_MOD_TLS_TIS_STATIC_PARAMS << 24)); + cseg->opmod_idx_opcode = cpu_to_be32((pc << 8) | MLX5_OPCODE_UMR | (opmod << 24)); cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) | STATIC_PARAMS_DS_CNT); cseg->fm_ce_se = fence ? MLX5_FENCE_MODE_INITIATOR_SMALL : 0; @@ -76,12 +78,15 @@ mlx5e_ktls_build_static_params(struct mlx5e_set_tls_static_params_wqe *wqe, } static void -fill_progress_params(struct mlx5_wqe_tls_progress_params_seg *params, u32 tis_tir_num) +fill_progress_params(struct mlx5_wqe_tls_progress_params_seg *params, u32 tis_tir_num, + u32 next_record_tcp_sn) { u8 *ctx = params->ctx; params->tis_tir_num = cpu_to_be32(tis_tir_num); + MLX5_SET(tls_progress_params, ctx, next_record_tcp_sn, + next_record_tcp_sn); MLX5_SET(tls_progress_params, ctx, record_tracker_state, MLX5E_TLS_PROGRESS_PARAMS_RECORD_TRACKER_STATE_START); MLX5_SET(tls_progress_params, ctx, auth_state, @@ -92,19 +97,22 @@ void mlx5e_ktls_build_progress_params(struct mlx5e_set_tls_progress_params_wqe *wqe, u16 pc, u32 sqn, u32 tis_tir_num, bool fence, + u32 next_record_tcp_sn, enum tls_offload_ctx_dir direction) { struct mlx5_wqe_ctrl_seg *cseg = &wqe->ctrl; + u8 opmod = direction == TLS_OFFLOAD_CTX_DIR_TX ? + MLX5_OPC_MOD_TLS_TIS_PROGRESS_PARAMS : + MLX5_OPC_MOD_TLS_TIR_PROGRESS_PARAMS; #define PROGRESS_PARAMS_DS_CNT DIV_ROUND_UP(sizeof(*wqe), MLX5_SEND_WQE_DS) cseg->opmod_idx_opcode = - cpu_to_be32((pc << 8) | MLX5_OPCODE_SET_PSV | - (MLX5_OPC_MOD_TLS_TIS_PROGRESS_PARAMS << 24)); + cpu_to_be32((pc << 8) | MLX5_OPCODE_SET_PSV | (opmod << 24)); cseg->qpn_ds = cpu_to_be32((sqn << MLX5_WQE_CTRL_QPN_SHIFT) | PROGRESS_PARAMS_DS_CNT); cseg->fm_ce_se = fence ? MLX5_FENCE_MODE_INITIATOR_SMALL : 0; - fill_progress_params(&wqe->params, tis_tir_num); + fill_progress_params(&wqe->params, tis_tir_num, next_record_tcp_sn); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.h index 7ce6b1c41d9b..7bdd6ec6c981 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.h @@ -19,6 +19,10 @@ u16 mlx5e_ktls_get_stop_room(struct mlx5e_txqsq *sq); bool mlx5e_ktls_handle_tx_skb(struct tls_context *tls_ctx, struct mlx5e_txqsq *sq, struct sk_buff *skb, int datalen, struct mlx5e_accel_tx_tls_state *state); +void mlx5e_ktls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, + struct mlx5_cqe64 *cqe, u32 *cqe_bcnt); + +void mlx5e_ktls_handle_ctx_completion(struct mlx5e_icosq_wqe_info *wi); void mlx5e_ktls_tx_handle_resync_dump_comp(struct mlx5e_txqsq *sq, struct mlx5e_tx_wqe_info *wi, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h index d1d747cb2dcb..566cf24eb0fe 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_utils.h @@ -23,6 +23,9 @@ enum { int mlx5e_ktls_add_tx(struct net_device *netdev, struct sock *sk, struct tls_crypto_info *crypto_info, u32 start_offload_tcp_sn); void mlx5e_ktls_del_tx(struct net_device *netdev, struct tls_context *tls_ctx); +int mlx5e_ktls_add_rx(struct net_device *netdev, struct sock *sk, + struct tls_crypto_info *crypto_info, u32 start_offload_tcp_sn); +void mlx5e_ktls_del_rx(struct net_device *netdev, struct tls_context *tls_ctx); struct mlx5e_set_tls_static_params_wqe { struct mlx5_wqe_ctrl_seg ctrl; @@ -64,6 +67,7 @@ void mlx5e_ktls_build_progress_params(struct mlx5e_set_tls_progress_params_wqe *wqe, u16 pc, u32 sqn, u32 tis_tir_num, bool fence, + u32 next_record_tcp_sn, enum tls_offload_ctx_dir direction); #endif /* __MLX5E_TLS_UTILS_H__ */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c index c01c17a5c6de..99beb928feff 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.c @@ -197,6 +197,7 @@ void mlx5e_tls_build_netdev(struct mlx5e_priv *priv) return; } + /* FPGA */ if (!mlx5_accel_is_tls_device(priv->mdev)) return; @@ -221,11 +222,19 @@ void mlx5e_tls_build_netdev(struct mlx5e_priv *priv) int mlx5e_tls_init(struct mlx5e_priv *priv) { - struct mlx5e_tls *tls = kzalloc(sizeof(*tls), GFP_KERNEL); + struct mlx5e_tls *tls; + if (!mlx5_accel_is_tls_device(priv->mdev)) + return 0; + + tls = kzalloc(sizeof(*tls), GFP_KERNEL); if (!tls) return -ENOMEM; + tls->rx_wq = create_singlethread_workqueue("mlx5e_tls_rx"); + if (!tls->rx_wq) + return -ENOMEM; + priv->tls = tls; return 0; } @@ -237,6 +246,7 @@ void mlx5e_tls_cleanup(struct mlx5e_priv *priv) if (!tls) return; + destroy_workqueue(tls->rx_wq); kfree(tls); priv->tls = NULL; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h index 9015f3f7792d..ca0c2ebb41a1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h @@ -53,6 +53,7 @@ struct mlx5e_tls_sw_stats { struct mlx5e_tls { struct mlx5e_tls_sw_stats sw_stats; + struct workqueue_struct *rx_wq; }; struct mlx5e_tls_offload_context_tx { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c index 4d796fea906d..182841322ce4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c @@ -278,9 +278,10 @@ bool mlx5e_tls_handle_tx_skb(struct net_device *netdev, struct mlx5e_txqsq *sq, if (WARN_ON_ONCE(tls_ctx->netdev != netdev)) goto err_out; - if (MLX5_CAP_GEN(sq->channel->mdev, tls_tx)) + if (mlx5_accel_is_ktls_tx(sq->channel->mdev)) return mlx5e_ktls_handle_tx_skb(tls_ctx, sq, skb, datalen, state); + /* FPGA */ skb_seq = ntohl(tcp_hdr(skb)->seq); context = mlx5e_get_tls_tx_context(tls_ctx); expected_seq = context->expected_seq; @@ -354,12 +355,16 @@ static int tls_update_resync_sn(struct net_device *netdev, return 0; } -void mlx5e_tls_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb, - u32 *cqe_bcnt) +void mlx5e_tls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, + struct mlx5_cqe64 *cqe, u32 *cqe_bcnt) { struct mlx5e_tls_metadata *mdata; struct mlx5e_priv *priv; + if (likely(mlx5_accel_is_ktls_rx(rq->mdev))) + return mlx5e_ktls_handle_rx_skb(rq, skb, cqe, cqe_bcnt); + + /* FPGA */ if (!is_metadata_hdr_valid(skb)) return; @@ -370,13 +375,13 @@ void mlx5e_tls_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb, skb->decrypted = 1; break; case SYNDROM_RESYNC_REQUEST: - tls_update_resync_sn(netdev, skb, mdata); - priv = netdev_priv(netdev); + tls_update_resync_sn(rq->netdev, skb, mdata); + priv = netdev_priv(rq->netdev); atomic64_inc(&priv->tls->sw_stats.rx_tls_resync_request); break; case SYNDROM_AUTH_FAILED: /* Authentication failure will be observed and verified by kTLS */ - priv = netdev_priv(netdev); + priv = netdev_priv(rq->netdev); atomic64_inc(&priv->tls->sw_stats.rx_tls_auth_fail); break; default: @@ -395,9 +400,10 @@ u16 mlx5e_tls_get_stop_room(struct mlx5e_txqsq *sq) if (!mlx5_accel_is_tls_device(mdev)) return 0; - if (MLX5_CAP_GEN(mdev, tls_tx)) + if (mlx5_accel_is_ktls_device(mdev)) return mlx5e_ktls_get_stop_room(sq); + /* FPGA */ /* Resync SKB. */ return mlx5e_stop_room_for_wqe(MLX5_SEND_WQE_MAX_WQEBBS); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h index 2a7b98531539..8bb790674042 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h @@ -49,8 +49,8 @@ bool mlx5e_tls_handle_tx_skb(struct net_device *netdev, struct mlx5e_txqsq *sq, void mlx5e_tls_handle_tx_wqe(struct mlx5e_txqsq *sq, struct mlx5_wqe_ctrl_seg *cseg, struct mlx5e_accel_tx_tls_state *state); -void mlx5e_tls_handle_rx_skb(struct net_device *netdev, struct sk_buff *skb, - u32 *cqe_bcnt); +void mlx5e_tls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, + struct mlx5_cqe64 *cqe, u32 *cqe_bcnt); #else diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 11997c23dfb5..0f1578a5e538 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -1441,6 +1441,7 @@ void mlx5e_close_icosq(struct mlx5e_icosq *sq) struct mlx5e_channel *c = sq->channel; mlx5e_destroy_sq(c->mdev, sq->sqn); + mlx5e_free_icosq_descs(sq); mlx5e_free_icosq(sq); } @@ -3853,6 +3854,7 @@ int mlx5e_set_features(struct net_device *netdev, netdev_features_t features) #ifdef CONFIG_MLX5_EN_ARFS err |= MLX5E_HANDLE_FEATURE(NETIF_F_NTUPLE, set_feature_arfs); #endif + err |= MLX5E_HANDLE_FEATURE(NETIF_F_HW_TLS_RX, mlx5e_ktls_set_feature_rx); if (err) { netdev->features = oper_features; @@ -5143,8 +5145,14 @@ static int mlx5e_init_nic_rx(struct mlx5e_priv *priv) if (err) goto err_destroy_flow_steering; + err = mlx5e_accel_init_rx(priv); + if (err) + goto err_tc_nic_cleanup; + return 0; +err_tc_nic_cleanup: + mlx5e_tc_nic_cleanup(priv); err_destroy_flow_steering: mlx5e_destroy_flow_steering(priv); err_destroy_xsk_tirs: @@ -5168,6 +5176,7 @@ static int mlx5e_init_nic_rx(struct mlx5e_priv *priv) static void mlx5e_cleanup_nic_rx(struct mlx5e_priv *priv) { + mlx5e_accel_cleanup_rx(priv); mlx5e_tc_nic_cleanup(priv); mlx5e_destroy_flow_steering(priv); mlx5e_destroy_direct_tirs(priv, priv->xsk_tir); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index dbb1c6323967..9a6958acf87d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -578,6 +578,30 @@ bool mlx5e_post_rx_wqes(struct mlx5e_rq *rq) return !!err; } +void mlx5e_free_icosq_descs(struct mlx5e_icosq *sq) +{ + u16 sqcc; + + sqcc = sq->cc; + + while (sqcc != sq->pc) { + struct mlx5e_icosq_wqe_info *wi; + u16 ci; + + ci = mlx5_wq_cyc_ctr2ix(&sq->wq, sqcc); + wi = &sq->db.wqe_info[ci]; + sqcc += wi->num_wqebbs; +#ifdef CONFIG_MLX5_EN_TLS + switch (wi->wqe_type) { + case MLX5E_ICOSQ_WQE_SET_PSV_TLS: + mlx5e_ktls_handle_ctx_completion(wi); + break; + } +#endif + } + sq->cc = sqcc; +} + int mlx5e_poll_ico_cq(struct mlx5e_cq *cq) { struct mlx5e_icosq *sq = container_of(cq, struct mlx5e_icosq, cq); @@ -633,6 +657,13 @@ int mlx5e_poll_ico_cq(struct mlx5e_cq *cq) break; case MLX5E_ICOSQ_WQE_NOP: break; +#ifdef CONFIG_MLX5_EN_TLS + case MLX5E_ICOSQ_WQE_UMR_TLS: + break; + case MLX5E_ICOSQ_WQE_SET_PSV_TLS: + mlx5e_ktls_handle_ctx_completion(wi); + break; +#endif default: netdev_WARN_ONCE(cq->channel->netdev, "Bad WQE type in ICOSQ WQE info: 0x%x\n", @@ -983,7 +1014,7 @@ static inline void mlx5e_build_rx_skb(struct mlx5_cqe64 *cqe, skb->mac_len = ETH_HLEN; #ifdef CONFIG_MLX5_EN_TLS - mlx5e_tls_handle_rx_skb(netdev, skb, &cqe_bcnt); + mlx5e_tls_handle_rx_skb(rq, skb, cqe, &cqe_bcnt); #endif if (lro_num_seg > 1) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw.c b/drivers/net/ethernet/mellanox/mlx5/core/fw.c index a5fbe7343508..c3095863372c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw.c @@ -35,6 +35,7 @@ #include #include "mlx5_core.h" #include "../../mlxfw/mlxfw.h" +#include "accel/tls.h" enum { MCQS_IDENTIFIER_BOOT_IMG = 0x1, @@ -236,7 +237,7 @@ int mlx5_query_hca_caps(struct mlx5_core_dev *dev) return err; } - if (MLX5_CAP_GEN(dev, tls_tx)) { + if (mlx5_accel_is_ktls_tx(dev) || mlx5_accel_is_ktls_rx(dev)) { err = mlx5_core_get_caps(dev, MLX5_CAP_TLS); if (err) return err; From patchwork Sat Jun 27 21:17:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216971 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57139C433DF for ; Sat, 27 Jun 2020 21:19:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2C356206D7 for ; Sat, 27 Jun 2020 21:19:16 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="RYzJFev4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726767AbgF0VTP (ORCPT ); Sat, 27 Jun 2020 17:19:15 -0400 Received: from mail-eopbgr60076.outbound.protection.outlook.com ([40.107.6.76]:16519 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725916AbgF0VTN (ORCPT ); Sat, 27 Jun 2020 17:19:13 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=IZ1IMC6umRLHLID3+wZJPtRdpZJM93RVVtz+UBFU/s1EUg7h4/y2jqvQicCAAwUQ7VHqr+ih80kGG5uVgUyel55hCoqE8s+kIbJR6FZ07Dg8E/baKyDimv9LCTDI/CfBxPaBWxHBgk64Zq0tON+d9vAQ0SyW+AF26cZVR/jd31/0+4OoxY1kzFFIw8ZYves7MgX+y+c0N00oMv9gZggrFU+7+eEaZYTd+GdTTpI70REbQ2TbJdKb8n5UfUU/5jbCea/3IOgI30wPo6ARs8FPN0fMiZMCYKPQxsftEdt71f9Nt9FORq4Ai2HZFAIwTfaZwQl3szYFpA4QsblmrX1oCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=11U0Wzn5YnOsqnyB/WwZylV16agrSaYo4uoTLZB69zg=; b=ZZs9RZupjHpfcTufhvwHhVQP+/oXT/fx7F2caGWuGw7BwuPvvj6DU5kNRxmno9fhk+wa2gZ8LQMa+QBTx3vL5MrCemURTJrffRfUY6+NhSJVZc45w28N6uw08st/XXRWXETUX7Kr6ewJxcpZcTC+yFfzwdC5OAMZYCH/b0gCfUTk2ey15b4p8UHuYyqfBBbCGY2EliJEWiYpSaIwluOCwhX7I47dplkmlmcnJiQ/cysTa+3hse7ir+dsc63cdXhvtABBQqN6jTA3F57d/ftCMRWOz9X2nzj76VbCGkPGAS6VXEbDsWkyoFZ5vw4iY9gT0017avNz1o5hpKEnQ70oEA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=11U0Wzn5YnOsqnyB/WwZylV16agrSaYo4uoTLZB69zg=; b=RYzJFev40f4Xqa9btpIwJljoZ0aO0fo3sNx8RLcJiL58O/laAsutcjvLsClKiuHaJ670L5jiQWeX4eMAHG5MWlgtnjtiAmHghBxgjmSQJL1hQmP2C/v2YI3uYXcsen1IK/dbByLFcq0UURJirvQYGnW6W03hFgoAIF+2ZIZOMm4= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:18:56 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:18:56 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Boris Pismenny , Tariq Toukan , Maxim Mikityanskiy , Saeed Mahameed Subject: [net-next 09/15] Revert "net/tls: Add force_resync for driver resync" Date: Sat, 27 Jun 2020 14:17:21 -0700 Message-Id: <20200627211727.259569-10-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:18:54 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: bea5ec3c-0397-4db9-b2f1-08d81adfb31b X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:364; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5HPKQB3ZEq17BhLHjgfdd7r0oX2Hln0qLJaDN2ipkpEaVmlTgnSNlpX72WiCSXLii0fImiGjMNeGQYEmNiPoxvLSuepQtn2wfexzIDAGRQp63Vsa1N34dnJ4oiDVOIw9+9wj+aWuxQw22BwckDlUj0pR6xZIkz9gCxsOfz2KLUyLsyM9vDGGewnBoCHGqBD3wdZKqWA9d+W+ZueT1ADyPRO5IRUNStXtROgtJ5EenBF7IGwOY8Qby+7DkEF8JDT+0iCU3iJYr0VyQ3kVeHwGX3S4/NnFBAkyltZ+IpBHIy18TMIheigCCu2aVTFkbRMrFZqSA+fOhaYIUXclF1NFG9/hTSFfvbVlMMJzXmo/ce+sggVCLt5zfeCirvNhko9r X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: NIDeV7OyCCNLekFJfGz2bj0QNZfKUklQU3DJdz7GGlUkY2iOmgQNtPHnDxRU2I56mUBtnlYOAEjF45QfH7iLFdRlv3gABLh9VkHKvlbh1CtR1AN1+baDuZxR07gc5STtlxciVjryn/VPHgF1goERMUyrtFlD7LNMmJOtYOaY+fTxh1gnrJBGehu3ILGx88wkbZUJk3X/eo1DNuDrhhSGhHPx+15mnuq8v4Bse4UFmhbXdljSrONxDDRNGI1epsxUPi1y6N4GtFzhyNKNX/uFSmLYkab+ikkJ635yWn8RYekPJjCjQcr+EzR4+SLRtkMqpGBSvd+ntiT5P47RaWNY03TFQgmj7ewIsaR50AjLAwJ7rcZ67lOE9tr6nP1QRwdpmgd8K56Cne5yXU8F02uVl4+ljRLyzXseCdJN3YMN9sbttsgWD+zO8jOdspOkHLSALVewQ1au8E8T+nzSuFAGh3Qpp7KYFUFCsz4zLDC8Z5M= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: bea5ec3c-0397-4db9-b2f1-08d81adfb31b X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:18:56.0140 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FoSUx54c3qyUXf7vkIJQDCF+uStMeOv+JDJW+ReEFg4M+O3vqQ+AHguSlcanS72MW4BDt2URhRrOnL5V0jvhhw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Boris Pismenny This reverts commit b3ae2459f89773adcbf16fef4b68deaaa3be1929. Revert the force resync API. Not in use. To be replaced by a better async resync API downstream. Signed-off-by: Boris Pismenny Signed-off-by: Tariq Toukan Reviewed-by: Maxim Mikityanskiy Signed-off-by: Saeed Mahameed --- include/net/tls.h | 12 +----------- net/tls/tls_device.c | 9 +++------ 2 files changed, 4 insertions(+), 17 deletions(-) diff --git a/include/net/tls.h b/include/net/tls.h index 3212d3c214a9..ca5f7f437289 100644 --- a/include/net/tls.h +++ b/include/net/tls.h @@ -607,22 +607,12 @@ tls_driver_ctx(const struct sock *sk, enum tls_offload_ctx_dir direction) #endif /* The TLS context is valid until sk_destruct is called */ -#define RESYNC_REQ (1 << 0) -#define RESYNC_REQ_FORCE (1 << 1) static inline void tls_offload_rx_resync_request(struct sock *sk, __be32 seq) { struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_offload_context_rx *rx_ctx = tls_offload_ctx_rx(tls_ctx); - atomic64_set(&rx_ctx->resync_req, ((u64)ntohl(seq) << 32) | RESYNC_REQ); -} - -static inline void tls_offload_rx_force_resync_request(struct sock *sk) -{ - struct tls_context *tls_ctx = tls_get_ctx(sk); - struct tls_offload_context_rx *rx_ctx = tls_offload_ctx_rx(tls_ctx); - - atomic64_set(&rx_ctx->resync_req, RESYNC_REQ | RESYNC_REQ_FORCE); + atomic64_set(&rx_ctx->resync_req, ((u64)ntohl(seq) << 32) | 1); } static inline void diff --git a/net/tls/tls_device.c b/net/tls/tls_device.c index 0e55f8365ce2..a562ebaaa33c 100644 --- a/net/tls/tls_device.c +++ b/net/tls/tls_device.c @@ -694,11 +694,10 @@ void tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq) { struct tls_context *tls_ctx = tls_get_ctx(sk); struct tls_offload_context_rx *rx_ctx; - bool is_req_pending, is_force_resync; u8 rcd_sn[TLS_MAX_REC_SEQ_SIZE]; + u32 sock_data, is_req_pending; struct tls_prot_info *prot; s64 resync_req; - u32 sock_data; u32 req_seq; if (tls_ctx->rx_conf != TLS_HW) @@ -713,11 +712,9 @@ void tls_device_rx_resync_new_rec(struct sock *sk, u32 rcd_len, u32 seq) resync_req = atomic64_read(&rx_ctx->resync_req); req_seq = resync_req >> 32; seq += TLS_HEADER_SIZE - 1; - is_req_pending = resync_req & RESYNC_REQ; - is_force_resync = resync_req & RESYNC_REQ_FORCE; + is_req_pending = resync_req; - if (likely(!is_req_pending) || - (!is_force_resync && req_seq != seq) || + if (likely(!is_req_pending) || req_seq != seq || !atomic64_try_cmpxchg(&rx_ctx->resync_req, &resync_req, 0)) return; break; From patchwork Sat Jun 27 21:17:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C72C9C433E1 for ; Sat, 27 Jun 2020 21:19:25 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A50FB2071A for ; Sat, 27 Jun 2020 21:19:25 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="agwUeSV6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726836AbgF0VTY (ORCPT ); Sat, 27 Jun 2020 17:19:24 -0400 Received: from mail-eopbgr60076.outbound.protection.outlook.com ([40.107.6.76]:16519 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725823AbgF0VTX (ORCPT ); Sat, 27 Jun 2020 17:19:23 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Iq84bjkfcwAiB1fOWOYaMpUjhhT1wRxaRclAT/X0S9WaVNJ9QJsZCfqZ9xL/9eam7Q7r9ANOK1YhiBHWONudGQU/wZFytSfac5utMcrar+NaFBDvZGYESCMDuC5VV4OxryFtChW86tl611G/6bcFm78wA4kgsUsVz9fBZSeqWyXpnxwQgAU2eD/RCrLo5u79uAGfP3C2CTf1QjrZMPkqtc7IigEAUxTinhzqMG2M1oKP8W/Iv8xtHl1Shi3hEwtOLTnLp/Kj/BDd9hW+eK4CwlxDVFqkukNCsN5u59+TjHdDoR/7WDZ+EQZBlQApG4oT7EW00/seXCNa1VXFF/DrnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yyuXvneBGq6Ns7v3rRuvPjOtsIMicDUWDRRfWGDIBfI=; b=oWvoveKDNqWj152w4pPPygHLY+ZTZ3yl6AaVvZuBr59la1APFouUn95X+tNRomzM/nbgN5cmbzlm7R9RBY1NL1zGZOo+mILgYzi8L0HTdGvyyJja7zewLmuW/YWetvtIPPxIO+VsnVbDv0lH6Vn06XZ3sggNFAMRA0YQd0mpfLnIQvx+UmOt+qWJrvmjE9BXhs1clobzn860Zz+DWaFW63n/3nblvuZSjM9eIuLioBLNhNH/gGKVg/56f9+dV0HLPX7hO4WOakhf81rc2jTU+nid2y9G2aQko8a9soe11xnSUJslsfhpskgSfD9i1oj+ylY8VKPQzWIMqb40r/wQJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yyuXvneBGq6Ns7v3rRuvPjOtsIMicDUWDRRfWGDIBfI=; b=agwUeSV6bXkQB/vBf4NBWugSDxgOwiYpenlShGbrkibUfeipsFlh7sWDSRyD+Aht3xp/oMxPPRUqmXhz32o/xA6/OTBF8WuYwzwDrrCEe8zlQ4dLWpapUIVIPtQSldNI7pfu0O9qeglhpw1+R6fMQ2E9/IjnTdiTtAYYFkFdVNw= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:19:04 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:19:04 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Tariq Toukan , Maxim Mikityanskiy , Saeed Mahameed Subject: [net-next 13/15] net/mlx5e: Increase Async ICO SQ size Date: Sat, 27 Jun 2020 14:17:25 -0700 Message-Id: <20200627211727.259569-14-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:19:02 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: cc973770-3ee1-4ab4-27a0-08d81adfb84c X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3631; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: I9Y8WEc3sk0hjFh6yDM9+Hv3xr+leXVJNfWutENHfhf3b2jSRV54oeziybR01MtplIIVxXoK5zymkEoo5Y7w9T4LFL0af7jbZIwIisRB9RFdDhpyWF8AOibABKlK6dkzjma9Bk+TCNyU5eF9strvCJkPd9DQay7PB++qdPqjCSUM/h/ncQMYcnLnGc3MXrpu0l3TQegbGYMAevLSwUn5vznNcrnc4kKVwiuDX3EmzPux2n+Mpn+UaKUGwLarAPiqh/OtpQDmr/V6dC5EWW56wqTZ+kw6NUzj7L/kpYjlboyWmFKqct33Z5cQQV96wo1n7/ShJ/idZA8XDSjuZ2jgRhkQjxNzZmM5DepVVzwi6uOThyfgMO9jH6jvfvpAnwr2 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 8QEffPWrybGDrBlirVYhKURlqDxdjQqB+TlFA8ouy7u158NCCEuRLcghTo2CmiFVvdABAnRrXY9Jy1992pgUVdOHFRrjedMogv0N3b0jiftSmr941fxbPDmrvUPgHbKci7iJjlPQEIsehmvK8klb4QL7dArI+EJ9bSwssnrCU7bUoJH61eTq4feWEZY8ujpVJ0Rsnaqsbprr0WloYK7Wlhf/1axEFsX3SfP0Q7ZB8/j06Sm3COBMVYsvEOjXYwLJajXrZeBvu9jaPgmKMHnfDjlKy8HrGhinDOaOPglDJfFLOQcy1EE2NK5xBQYdaBpuiaDds9gRx+R8k1SzNcDNx/1WfvluukOAHkTJ5onJlBloPRC69vmJk4Cr9A6sD/lHRcN7kr/LOm2GDK6YLP0FKpFoUQyD05p47mAWnqrPxFiat4Rs5ZYe/fQbnsoF20Fx5beoA+ZSYmeAdhrRRMbJW0eEK0Jl6NWTgiFPMKUYaFA= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: cc973770-3ee1-4ab4-27a0-08d81adfb84c X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:19:04.7091 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: L5geChi7d9W3Jk2ub99nCFsnNdQ+iNFQjQaJT53jACg/JBXfgRbfwCs8BTPOSqeDzmIXGcGspNj8ImWGC+UZvg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tariq Toukan Resync communication with HW for kTLS RX is done via the async ICOSQs. kTLS RX resync requests might come in bursts. To improve the success chances for such bursts, use a larger ICOSQ. Signed-off-by: Tariq Toukan Reviewed-by: Maxim Mikityanskiy Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 0f1578a5e538..3e6fcd545d2c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -2306,6 +2306,14 @@ static u8 mlx5e_build_icosq_log_wq_sz(struct mlx5e_params *params, } } +static u8 mlx5e_build_async_icosq_log_wq_sz(struct net_device *netdev) +{ + if (netdev->hw_features & NETIF_F_HW_TLS_RX) + return MLX5E_PARAMS_DEFAULT_LOG_SQ_SIZE; + + return MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE; +} + static void mlx5e_build_channel_param(struct mlx5e_priv *priv, struct mlx5e_params *params, struct mlx5e_channel_param *cparam) @@ -2315,7 +2323,7 @@ static void mlx5e_build_channel_param(struct mlx5e_priv *priv, mlx5e_build_rq_param(priv, params, NULL, &cparam->rq); icosq_log_wq_sz = mlx5e_build_icosq_log_wq_sz(params, &cparam->rq); - async_icosq_log_wq_sz = MLX5E_PARAMS_MINIMUM_LOG_SQ_SIZE; + async_icosq_log_wq_sz = mlx5e_build_async_icosq_log_wq_sz(priv->netdev); mlx5e_build_sq_param(priv, params, &cparam->txq_sq); mlx5e_build_xdpsq_param(priv, params, &cparam->xdp_sq); From patchwork Sat Jun 27 21:17:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 216968 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, MSGID_FROM_MTA_HEADER, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 17705C433E0 for ; Sat, 27 Jun 2020 21:20:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DD38A20663 for ; Sat, 27 Jun 2020 21:20:05 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=Mellanox.com header.i=@Mellanox.com header.b="UU4fQECh" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726863AbgF0VUE (ORCPT ); Sat, 27 Jun 2020 17:20:04 -0400 Received: from mail-eopbgr60076.outbound.protection.outlook.com ([40.107.6.76]:16519 "EHLO EUR04-DB3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725823AbgF0VUE (ORCPT ); Sat, 27 Jun 2020 17:20:04 -0400 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=oWVEL+k4CIfwMangaK9kVp4HUa5TBWqb0gApl6ZtxuJ5wJin+Z2VCl1oH9jVgENd/TyjWneGIf2qrTk/YH30gFaFe62VHyXs1VmxWKtCA8MH137Gq0hZxHXPqUu2TaFBYuLfpjPp/gm+hqz11Z14Ir/Vc/smT6mo//zW7ZYAMUEX1UoUf1UQTr92jxbdUS926eUqh05ujOrOQ+AwmIKDqK/Vk3iUC3IOUnTBQvvI2DwrfhCr6h3XuY1DKsLpDRRSr6rhpPPfbySqvmHRaQouqg6CaU4xntsXx/lMgNcMcrCW+Li5DC5JVUGcW2fR41ScNb9qeRfl8bpZMenaxjY5sQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A69AU7lL/utccKpAaTl6jnDDLNvZniRkt7k73AuAhCo=; b=ofZ72yVn7hUesnb7LsgVDbSQ/wP0vEZS79Ookwng4aBQ/VuHXZH+SSgej9Mcbqp0qAHcdR5usfXyecDipY4J9kL3Rx+8iEN2TWwCGI6v4yH394NDwYfrXAiiWIVD8NEAjmDpkRr7XssluMGwHfcaxHbE6iGjIGhCaYHw41wNOcr/g7MyYYmimXnAW1djy6Rosx5zjO318shSI8Tuuijb9FW5QdLPQHzEttFqWmoOkyGQzA5SSRSPvOHJ/j8WoMm7Bg+l1xkLcOS8WplLY2DEPoeel9dh9qSSiNkfa+v8P3hfyWGr0chUboIswGD3iU70I/+VAPU60qM30Y6QJNyxhg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=mellanox.com; dmarc=pass action=none header.from=mellanox.com; dkim=pass header.d=mellanox.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Mellanox.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=A69AU7lL/utccKpAaTl6jnDDLNvZniRkt7k73AuAhCo=; b=UU4fQEChWaXfiW3WWIdNKqYyc3EnbsjGRQE/9kOJXZZuxGQ4du6qlKOdBEyIF0WxtryXNeT++SE6ZrPkYWC2N4IeI9PKJqfmYe+FUjBd1ji/1sJIS1ZhjiHAem+GIrrocRSz+m37DulhgadEPxerWIFVK1V+imDXkZ6GCMzaAOY= Authentication-Results: davemloft.net; dkim=none (message not signed) header.d=none; davemloft.net; dmarc=none action=none header.from=mellanox.com; Received: from VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) by VI1PR05MB5134.eurprd05.prod.outlook.com (2603:10a6:803:ad::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21; Sat, 27 Jun 2020 21:19:09 +0000 Received: from VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c]) by VI1PR05MB5102.eurprd05.prod.outlook.com ([fe80::2405:4594:97a:13c%2]) with mapi id 15.20.3131.026; Sat, 27 Jun 2020 21:19:09 +0000 From: Saeed Mahameed To: "David S. Miller" , kuba@kernel.org Cc: netdev@vger.kernel.org, Tariq Toukan , Saeed Mahameed Subject: [net-next 15/15] net/mlx5e: kTLS, Improve rx handler function call Date: Sat, 27 Jun 2020 14:17:27 -0700 Message-Id: <20200627211727.259569-16-saeedm@mellanox.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20200627211727.259569-1-saeedm@mellanox.com> References: <20200627211727.259569-1-saeedm@mellanox.com> X-ClientProxiedBy: BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) To VI1PR05MB5102.eurprd05.prod.outlook.com (2603:10a6:803:5e::23) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from smtp.office365.com (73.15.39.150) by BY5PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:1e0::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3131.21 via Frontend Transport; Sat, 27 Jun 2020 21:19:07 +0000 X-Mailer: git-send-email 2.26.2 X-Originating-IP: [73.15.39.150] X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 81a43319-816a-48fb-1573-08d81adfbaec X-MS-TrafficTypeDiagnostic: VI1PR05MB5134: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:669; X-Forefront-PRVS: 0447DB1C71 X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: BqBKjbb2eBImvaMXjtiVbziJueUuzYGdT3Zl2vZ2O2xB1Aso+dHluXyH4n3m+IDH4FVPnXagoXImhn/sv8foGugroZws6DutDF4SjfZdzEUYZ+d1/7joDDtncKD3BGTBD/0O0X4l0bzWVCl8NQputNfHnQYU8yjC9zS3YIaZdBRWO9w6IUoX+3eItKV2zQDnLHpDqnu3RSfqUfQ1Y0ZZX/J6yEdxPc/yfzSZiTmd7ZCbIt0R1x1ChKyg3bWSQbSVNGqxweDQl2StYk3KdI05QMusbKgSo5XzDxyqOgfF3mm/tBLeiZFuWOocRKK4jR02DW98TNqxbrmZhPxlwUZo6gyYHW2a5oVxB5aVmWLBSwrsohGZwq2rs4QV7rDNIxH8 X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:VI1PR05MB5102.eurprd05.prod.outlook.com; PTR:; CAT:NONE; SFTY:; SFS:(4636009)(366004)(376002)(136003)(39850400004)(346002)(396003)(66476007)(107886003)(54906003)(478600001)(4326008)(316002)(6512007)(16526019)(6506007)(2616005)(1076003)(6486002)(956004)(6666004)(66556008)(8936002)(52116002)(86362001)(36756003)(5660300002)(66946007)(26005)(8676002)(83380400001)(2906002)(186003)(54420400002); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: UM/13nA04wwJRF9Oz/o/6TFI5H/bThxCGtnXVvGtPJ8HZEyrkvNHNYaXWFe5U311mB3XheF50p6PfwUJZwvU12RWXHpHiin7D6evY4q4qBsHoJCEBPZa2e5c8/0hjP9za/4O/szm94+vv0K2YRHb7MZY8np9RpkzKeG9mesVttzKr0jK8B6w0Gzv48Eq4EkflCtmyHBQRq7ddUZH1kpdWDT6Bx9M7moyTBzBkfinA3YSc2ij191J89WVGF3yJIFz8qFMN5Z7d68xSFCNYieAru1MjB6GLUnqLdpLGEKua4LH1qYrEjlYf3Ujrypl+j5xI3ZBFfpXTFxJ6ee5YkCRUaWT+Ws9oRLRH5CrQUC6lr8WSqJg7Yt6jscBaWSCuHRG83hM9xz71ZcPGTAVK+SF1foxTkGiazYGRpGFY/OGp9MRc61dNNogZ4ghvlFn7kNfjYopCAZ0OOBvrzOvYN/LZowI3sCA60EWVR7qxNzt4DY= X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-Network-Message-Id: 81a43319-816a-48fb-1573-08d81adfbaec X-MS-Exchange-CrossTenant-AuthSource: VI1PR05MB5102.eurprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2020 21:19:09.1007 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0lVZiyApvuCgpwIgd3je5TBwIUcFaNTXAy+gV+zKd04J12YZwo5/lxG2y7O5l8SP1S2qtalDC4PSvG5N31RDWA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR05MB5134 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: Tariq Toukan Prior to this patch mlx5e tls rx handler was called unconditionally on all rx frames and the decision whether a frame is a valid tls record is done inside that function. A function call can be expensive especially for regular rx packet rate. To avoid this, check the tls validity before jumping into the tls rx handler. While at it, split between kTLS device offload rx handler and FPGA tls rx handler using a similar method. Signed-off-by: Saeed Mahameed Signed-off-by: Tariq Toukan --- drivers/net/ethernet/mellanox/mlx5/core/en.h | 1 + .../mellanox/mlx5/core/en_accel/ktls_rx.c | 10 ++------- .../mellanox/mlx5/core/en_accel/tls.h | 6 ++++++ .../mellanox/mlx5/core/en_accel/tls_rxtx.c | 12 +++-------- .../mellanox/mlx5/core/en_accel/tls_rxtx.h | 21 +++++++++++++++++-- .../net/ethernet/mellanox/mlx5/core/en_main.c | 3 +++ .../net/ethernet/mellanox/mlx5/core/en_rx.c | 2 -- 7 files changed, 34 insertions(+), 21 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index ec5bf73f9b07..2957edb7e0b7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -263,6 +263,7 @@ enum { MLX5E_RQ_STATE_AM, MLX5E_RQ_STATE_NO_CSUM_COMPLETE, MLX5E_RQ_STATE_CSUM_FULL, /* cqe_csum_full hw bit is set */ + MLX5E_RQ_STATE_FPGA_TLS, /* FPGA TLS enabled */ }; struct mlx5e_cq { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c index e0a8f9d63b30..d7215defd403 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c @@ -517,15 +517,9 @@ void mlx5e_ktls_rx_resync(struct net_device *netdev, struct sock *sk, void mlx5e_ktls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, struct mlx5_cqe64 *cqe, u32 *cqe_bcnt) { - u8 tls_offload = get_cqe_tls_offload(cqe); - struct mlx5e_rq_stats *stats; - - if (likely(tls_offload == CQE_TLS_OFFLOAD_NOT_DECRYPTED)) - return; - - stats = rq->stats; + struct mlx5e_rq_stats *stats = rq->stats; - switch (tls_offload) { + switch (get_cqe_tls_offload(cqe)) { case CQE_TLS_OFFLOAD_DECRYPTED: skb->decrypted = 1; stats->tls_decrypted_packets++; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h index ca0c2ebb41a1..bd270a85c804 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h @@ -87,6 +87,11 @@ mlx5e_get_tls_rx_context(struct tls_context *tls_ctx) base); } +static inline bool mlx5e_is_tls_on(struct mlx5e_priv *priv) +{ + return priv->tls; +} + void mlx5e_tls_build_netdev(struct mlx5e_priv *priv); int mlx5e_tls_init(struct mlx5e_priv *priv); void mlx5e_tls_cleanup(struct mlx5e_priv *priv); @@ -103,6 +108,7 @@ static inline void mlx5e_tls_build_netdev(struct mlx5e_priv *priv) mlx5e_ktls_build_netdev(priv); } +static inline bool mlx5e_is_tls_on(struct mlx5e_priv *priv) { return false; } static inline int mlx5e_tls_init(struct mlx5e_priv *priv) { return 0; } static inline void mlx5e_tls_cleanup(struct mlx5e_priv *priv) { } static inline int mlx5e_tls_get_count(struct mlx5e_priv *priv) { return 0; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c index 182841322ce4..b0c31d49ff8d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c @@ -355,19 +355,13 @@ static int tls_update_resync_sn(struct net_device *netdev, return 0; } -void mlx5e_tls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, - struct mlx5_cqe64 *cqe, u32 *cqe_bcnt) +/* FPGA tls rx handler */ +void mlx5e_tls_handle_rx_skb_metadata(struct mlx5e_rq *rq, struct sk_buff *skb, + u32 *cqe_bcnt) { struct mlx5e_tls_metadata *mdata; struct mlx5e_priv *priv; - if (likely(mlx5_accel_is_ktls_rx(rq->mdev))) - return mlx5e_ktls_handle_rx_skb(rq, skb, cqe, cqe_bcnt); - - /* FPGA */ - if (!is_metadata_hdr_valid(skb)) - return; - /* Use the metadata */ mdata = (struct mlx5e_tls_metadata *)(skb->data + ETH_HLEN); switch (mdata->content.recv.syndrome) { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h index 8bb790674042..5f162ad2ee8f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h @@ -34,6 +34,7 @@ #ifndef __MLX5E_TLS_RXTX_H__ #define __MLX5E_TLS_RXTX_H__ +#include "accel/accel.h" #include "en_accel/ktls_txrx.h" #ifdef CONFIG_MLX5_EN_TLS @@ -49,11 +50,27 @@ bool mlx5e_tls_handle_tx_skb(struct net_device *netdev, struct mlx5e_txqsq *sq, void mlx5e_tls_handle_tx_wqe(struct mlx5e_txqsq *sq, struct mlx5_wqe_ctrl_seg *cseg, struct mlx5e_accel_tx_tls_state *state); -void mlx5e_tls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, - struct mlx5_cqe64 *cqe, u32 *cqe_bcnt); +void mlx5e_tls_handle_rx_skb_metadata(struct mlx5e_rq *rq, struct sk_buff *skb, + u32 *cqe_bcnt); + +static inline void +mlx5e_tls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, + struct mlx5_cqe64 *cqe, u32 *cqe_bcnt) +{ + if (unlikely(get_cqe_tls_offload(cqe))) /* cqe bit indicates a TLS device */ + return mlx5e_ktls_handle_rx_skb(rq, skb, cqe, cqe_bcnt); + + if (unlikely(test_bit(MLX5E_RQ_STATE_FPGA_TLS, &rq->state) && is_metadata_hdr_valid(skb))) + return mlx5e_tls_handle_rx_skb_metadata(rq, skb, cqe_bcnt); +} #else +static inline bool +mlx5e_accel_is_tls(struct mlx5_cqe64 *cqe, struct sk_buff *skb) { return false; } +static inline void +mlx5e_tls_handle_rx_skb(struct mlx5e_rq *rq, struct sk_buff *skb, + struct mlx5_cqe64 *cqe, u32 *cqe_bcnt) {} static inline u16 mlx5e_tls_get_stop_room(struct mlx5e_txqsq *sq) { return 0; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 3e6fcd545d2c..046cfb0ea180 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -873,6 +873,9 @@ int mlx5e_open_rq(struct mlx5e_channel *c, struct mlx5e_params *params, if (err) goto err_destroy_rq; + if (mlx5e_is_tls_on(c->priv) && !mlx5_accel_is_ktls_device(c->mdev)) + __set_bit(MLX5E_RQ_STATE_FPGA_TLS, &c->rq.state); /* must be FPGA */ + if (MLX5_CAP_ETH(c->mdev, cqe_checksum_full)) __set_bit(MLX5E_RQ_STATE_CSUM_FULL, &c->rq.state); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c index 4b7c119c8946..8b42f729a4f7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c @@ -1019,9 +1019,7 @@ static inline void mlx5e_build_rx_skb(struct mlx5_cqe64 *cqe, skb->mac_len = ETH_HLEN; -#ifdef CONFIG_MLX5_EN_TLS mlx5e_tls_handle_rx_skb(rq, skb, cqe, &cqe_bcnt); -#endif if (lro_num_seg > 1) { mlx5e_lro_update_hdr(skb, cqe, cqe_bcnt);