[API-NEXT,v2,2/3] api: ipsec: add inline IPSEC support

Added support for inline IPSEC processing on packet input and
output. Inline mode IPSEC and traffic manager cannot be enabled
(currently) on the same pktio interface.

Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

---
 include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++++++++---
 include/odp/api/spec/packet_io.h |  32 ++++
 2 files changed, 355 insertions(+), 25 deletions(-)

-- 
2.8.1

On Tue, Mar 21, 2017 at 9:17 AM, Petri Savolainen <
petri.savolainen@linaro.org> wrote:

> Added support for inline IPSEC processing on packet input and

> output. Inline mode IPSEC and traffic manager cannot be enabled

> (currently) on the same pktio interface.

>

> Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

> ---

>  include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++

> ++++++---

>  include/odp/api/spec/packet_io.h |  32 ++++

>  2 files changed, 355 insertions(+), 25 deletions(-)

>

> diff --git a/include/odp/api/spec/ipsec.h b/include/odp/api/spec/ipsec.h

> index e951e49..23d02cf 100644

> --- a/include/odp/api/spec/ipsec.h

> +++ b/include/odp/api/spec/ipsec.h

> @@ -19,6 +19,8 @@ extern "C" {

>  #endif

>

>  #include <odp/api/crypto.h>

> +#include <odp/api/packet_io.h>

> +#include <odp/api/classification.h>

>

>  /** @defgroup odp_ipsec ODP IPSEC

>   *  Operations of IPSEC API.

> @@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {

>           * Application uses asynchronous IPSEC operations,

>           * which return results via events.

>           */

> -       ODP_IPSEC_OP_MODE_ASYNC

> +       ODP_IPSEC_OP_MODE_ASYNC,

> +

> +       /** Inline IPSEC operation

> +         *

> +         * Packet input/output is connected directly to IPSEC

> inbound/outbound

> +         * processing. Application uses asynchronous or inline IPSEC

> +         * operations.

> +         */

> +       ODP_IPSEC_OP_MODE_INLINE,

> +

> +       /** IPSEC is disabled in inbound / outbound direction */

> +       ODP_IPSEC_OP_MODE_DISABLED

>

>  } odp_ipsec_op_mode_t;

>

>  /**

> + * Protocol layers in IPSEC configuration

> + */

> +typedef enum odp_ipsec_proto_layer_t {

> +       /** No layers */

> +       ODP_IPSEC_LAYER_NONE = 0,

> +

> +       /** Layer L2 protocols (Ethernet, VLAN, etc) */

> +       ODP_IPSEC_LAYER_L2,

> +

> +       /** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

>


Nit: Correct spelling is IPsec, not IPSec, per RFC 4301.


> +       ODP_IPSEC_LAYER_L3,

> +

> +       /** Layer L4 protocols (UDP, TCP, SCTP) */

> +       ODP_IPSEC_LAYER_L4,

> +

> +       /** All layers */

> +       ODP_IPSEC_LAYER_ALL

> +

> +} odp_ipsec_proto_layer_t;

> +

> +/**

>   * Configuration options for IPSEC inbound processing

>   */

>  typedef struct odp_ipsec_inbound_config_t {

> @@ -77,9 +111,113 @@ typedef struct odp_ipsec_inbound_config_t {

>                 uint32_t max;

>         } spi;

>

> +       /** Retain outer headers

> +        *

> +        *  Select up to which protocol layer (at least) outer headers are

> +        *  retained in inbound inline processing. Default value is

> +        *  ODP_IPSEC_LAYER_NONE.

> +        *

> +        *  ODP_IPSEC_LAYER_NONE: Application does not require any outer

> +        *                        headers to be retained.

> +        *

> +        *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.

> +        *

> +        *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3, otherwise

> the

> +        *                        same as ODP_IPSEC_LAYER_ALL.

> +        *

> +        *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4, otherwise

> the

> +        *                        same as ODP_IPSEC_LAYER_ALL.

> +        *

> +        *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before IPSEC

> are

> +        *                        retained. In transport mode, all headers

> +        *                        before IP (carrying IPSEC) are retained.

> +        *

> +        */

> +       odp_ipsec_proto_layer_t retain_outer;

> +

> +       /** Parse packet headers in IPSEC payload

> +        *

> +        *  Select header parsing level after inbound processing. Packet

> headers

> +        *  in IPSEC payload must be parsed (at least) up to this level.

> +        *  Default value is ODP_IPSEC_LAYER_NONE.

> +        *

> +        *  Note that IPSec payload is never a L2 packet

> (ODP_IPSEC_LAYER_L2

>


IPsec


> +        *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC payload

> +        *  starts after IP header (ODP_IPSEC_LAYER_L3 equals

> +        *  ODP_IPSEC_LAYER_NONE).

> +        */

> +       odp_ipsec_proto_layer_t parse;

> +

> +       /** Flags to control IPSEC payload data checks up to the selected

> parse

> +        *  level. */

> +       union {

> +               struct {

> +                       /** Check IPv4 header checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t ipv4_chksum   : 1;

> +

> +                       /** Check UDP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t udp_chksum    : 1;

> +

> +                       /** Check TCP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t tcp_chksum    : 1;

> +

> +                       /** Check SCTP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t sctp_chksum   : 1;

> +               } check;

> +

> +               /** All bits of the bit field structure

> +                 *

> +                 * This field can be used to set/clear all flags, or

> bitwise

> +                 * operations over the entire structure. */

> +               uint32_t all_check;

> +       };

> +

>  } odp_ipsec_inbound_config_t;

>

>  /**

> + * Configuration options for IPSEC outbound processing

> + */

> +typedef struct odp_ipsec_outbound_config_t {

> +       /** Flags to control L3/L4 checksum insertion as part of outbound

> +        *  packet processing. Packet must have set with valid L3/L4

> offsets.

> +        *  Checksum configuration is ignored for packets that checksum

> cannot

> +        *  be computed for (e.g. IPv4 fragments). Application may use a

> packet

> +        *  metadata flag to disable checksum insertion per packet bases.

> +        */

> +       union {

> +               struct {

> +                       /** Insert IPv4 header checksum on the payload

> packet

> +                        *  before IPSEC transformation. Default value is

> 0. */

> +                       uint32_t inner_ipv4   : 1;

> +

> +                       /** Insert UDP header checksum on the payload

> packet

> +                        *  before IPSEC transformation. Default value is

> 0. */

> +                       uint32_t inner_udp    : 1;

> +

> +                       /** Insert TCP header checksum on the payload

> packet

> +                        *  before IPSEC transformation. Default value is

> 0. */

> +                       uint32_t inner_tcp    : 1;

> +

> +                       /** Insert SCTP header checksum on the payload

> packet

> +                        *  before IPSEC transformation. Default value is

> 0. */

> +                       uint32_t inner_sctp   : 1;

> +

> +               } chksum;

> +

> +               /** All bits of the bit field structure

> +                 *

> +                 * This field can be used to set/clear all flags, or

> bitwise

> +                 * operations over the entire structure. */

> +               uint32_t all_chksum;

> +       };

> +

> +} odp_ipsec_outbound_config_t;

> +

> +/**

>   * IPSEC capability

>   */

>  typedef struct odp_ipsec_capability_t {

> @@ -102,6 +240,24 @@ typedef struct odp_ipsec_capability_t {

>          */

>         uint8_t op_mode_async;

>

> +       /** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE) support

> +        *

> +        *  0: Inline IPSEC operation is not supported

> +        *  1: Inline IPSEC operation is supported

> +        *  2: Inline IPSEC operation is supported and preferred

> +        */

> +       uint8_t op_mode_inline;

> +

> +       /** Support of inline classification (ODP_IPSEC_DEST_CLS) for

> resulting

> +        *  inbound packets.

> +        *

> +        *  0: Inline classification of resulting packets is not supported

> +        *  1: Inline classification of resulting packets is supported

> +        *  2: Inline classification of resulting packets is supported and

> +        *     preferred

> +        */

> +       uint8_t cls_inline;

> +

>         /** Soft expiry limit in seconds support

>          *

>          *  0: Limit is not supported

> @@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {

>   * IPSEC configuration options

>   */

>  typedef struct odp_ipsec_config_t {

> -       /** IPSEC operation mode. Application selects which mode (sync or

> async)

> -        *  will be used for IPSEC operations.

> +       /** Inbound IPSEC operation mode. Application selects which mode

> +        *  will be used for inbound IPSEC operations.

>          *

>          *  @see odp_ipsec_in(), odp_ipsec_in_enq()

>          */

> -       odp_ipsec_op_mode_t op_mode;

> +       odp_ipsec_op_mode_t inbound_mode;

> +

> +       /** Outbound IPSEC operation mode. Application selects which mode

> +        *  will be used for outbound IPSEC operations.

> +        *

> +        *  @see odp_ipsec_out(), odp_ipsec_out_enq(),

> odp_ipsec_out_inline()

> +        */

> +       odp_ipsec_op_mode_t outbound_mode;

>

>         /** Maximum number of IPSEC SAs that application will use

>          * simultaneously */

> @@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {

>         /** IPSEC inbound processing configuration */

>         odp_ipsec_inbound_config_t inbound;

>

> +       /** IPSEC outbound processing configuration */

> +       odp_ipsec_outbound_config_t outbound;

> +

>  } odp_ipsec_config_t;

>

>  /**

> @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

>         ODP_IPSEC_LOOKUP_DISABLED = 0,

>

>         /** Inbound SA lookup is enabled. Used SPI values must be unique.

> */

> -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> +

> +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> +         * destination IP address. Used SPI values must be unique. */

> +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

>

>  } odp_ipsec_lookup_mode_t;

>

>  /**

> + * Result event destination

> + */

> +typedef enum odp_ipsec_dest_mode_t {

> +       /** Destination for IPSEC result events is a queue. */

> +       ODP_IPSEC_DEST_QUEUE = 0,

> +

> +       /** Destination for IPSEC result events is the classifier.

> +        *  IPSEC capability 'cls_inline' determines if inline

> classification

> +        *  is supported. */

> +       ODP_IPSEC_DEST_CLS

> +

> +} odp_ipsec_dest_mode_t;

> +

> +/**

>   * IPSEC Security Association (SA) parameters

>   */

>  typedef struct odp_ipsec_sa_param_t {

> @@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {

>         /** SPI value */

>         uint32_t spi;

>

> +       /** Additional inbound SA lookup parameters. Values are considered

> +        *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */

> +       struct {

> +               /* v4 or v6 */

> +               uint8_t ip_version;

>


Is this an enum? If not perhaps doc this as 4 = IPv4, 6 = IPv6.


> +

> +               /* IP destination address (NETWORK ENDIAN) */

> +               void    *dst_addr;

> +

> +       } lookup_param;

> +

>         /** MTU for outbound IP fragmentation offload

>          *

>          *  This is the maximum length of IP packets that outbound IPSEC

> @@ -434,13 +629,32 @@ typedef struct odp_ipsec_sa_param_t {

>          */

>         uint32_t mtu;

>

> +       /** Select where IPSEC result events are sent

> +        *

> +        *  Asynchronous and inline modes generate result events. Select

> where

> +        *  those events are sent. Inbound SAs may choose between a queue

> or

> +        *  the classifier. Outbound SAs must define a queue always.

> +        *  The default value is ODP_IPSEC_DEST_QUEUE.

> +        */

> +       odp_ipsec_dest_mode_t dest_mode;

> +

>         /** Destination queue for IPSEC events

>          *

> -        *  Operations in asynchronous mode enqueue resulting events into

> -        *  this queue.

> +        *  Operations in asynchronous or inline mode enqueue resulting

> events

> +        *  into this queue.

>          */

>         odp_queue_t dest_queue;

>

> +       /** Classifier destination CoS for IPSEC result events

> +        *

> +        *  Result events for successfully decapsulated packets are sent to

> +        *  classification through this CoS. Other result events are sent

> to

> +        *  'dest_queue'. This field is considered only when 'dest_mode' is

> +        *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any

> pktio

> +        *  interface default CoS.

> +        */

> +       odp_cos_t dest_cos;

> +

>         /** User defined SA context pointer

>          *

>          *  User defined context pointer associated with the SA.

> @@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {

>                 uint32_t all_error;

>         };

>

> +       union {

> +               /** Status flags */

> +               struct {

> +                       /** Packet was processed in inline mode */

> +                       uint32_t inline_mode      : 1;

> +

> +               } flag;

> +

> +               /** All flag bits */

> +               uint32_t all_flag;

> +       };

> +

>  } odp_ipsec_op_status_t;

>

>  /**

> @@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {

>  } odp_ipsec_op_param_t;

>

>  /**

> + * Outbound inline IPSEC operation parameters

> + */

> +typedef struct odp_ipsec_inline_op_param_t {

> +       /** Packet output interface for inline output operation

> +        *

> +        *  Outbound inline IPSEC operation uses this packet IO interface

> to

> +        *  output the packet after a successful IPSEC transformation. The

> pktio

> +        *  must have been configured to operate in inline IPSEC mode.

> +        */

> +       odp_pktio_t pktio;

>


Shouldn't we have an option for this to be output to an odp_tm_queue_t
instead of a pktio?
Also, for output are there any controls over which odp_pktout_queue_t is
used, or is this undefined?


> +

> +       /** Outer headers for inline output operation

> +        *

> +        *  Outbound inline IPSEC operation uses this information to

> prepend

> +        *  outer headers to the IPSEC packet before sending it out.

> +        */

> +       struct {

> +               /** Points to first byte of outer headers to be copied in

> +                *  front of the outgoing IPSEC packet. Implementation

> copies

> +                *  the headers during odp_ipsec_out_inline() call. */

> +               uint8_t *ptr;

>


Should this be an odp_packet_t rather than a raw set of bytes? The
rationale for making this an odp_packet_t is that this could then be a
packet reference which can be used for (re)transmit tracking. For input
having this be raw bytes makes sense since the ODP implementation is
controlling their allocation and placement, however for output how is the
application going to allocate these? Presumably it would create a header
via odp_packet_alloc() and then use odp_packet_data() to get this address,
but that seems cumbersome compared to simply passing the odp_packet_t
directly. The alternative would seem to require that the application use
some sort of odp_shm_reserve() call, but that gets awkward and is far less
efficient than packet allocation, which is a fastpath operation.


> +

> +               /** Outer header length in bytes */

> +               uint32_t len;

> +       } outer_hdr;

> +

> +} odp_ipsec_inline_op_param_t;

> +

> +/**

>   * IPSEC operation result for a packet

>   */

>  typedef struct odp_ipsec_packet_result_t {

> @@ -752,6 +1007,23 @@ typedef struct odp_ipsec_packet_result_t {

>          */

>         odp_ipsec_sa_t sa;

>

> +       /** Packet outer header status before inbound inline processing.

> +        *  This is valid only when status.flag.inline_mode is set.

> +        */

> +       struct {

> +               /** Points to the first byte of retained outer headers.

> These

> +                *  headers are stored in a contiquous, per packet,

> +                *  implementation specific memory space. Since the memory

> space

> +                *  may overlap with e.g. packet head/tailroom, the content

> +                *  becomes invalid if packet data storage is modified in

> +                *  anyway. The memory space may not be sharable to other

> +                *  threads. */

> +               uint8_t *ptr;

> +

> +               /** Outer header length in bytes */

> +               uint32_t len;

> +       } outer_hdr;

> +

>  } odp_ipsec_packet_result_t;

>

>  /**

> @@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {

>          *  at least 'num_pkt' elements.

>          *

>          *  Each successfully transformed packet has a valid value for

> these

> -        *  meta-data:

> +        *  meta-data regardless of the inner packet parse configuration.

>


Vestigial typo. It's metadata, not meta-data. Might as well clean up here.


> +        *  (odp_ipsec_inbound_config_t):

>          *  * L3 offset: Offset to the first byte of the (outmost) IP

> header

> -        *  * L4 offset: Offset to the first byte of the valid and known L4

> -        *               header (immediately following the IP header).

> -        *  * Various flags about L3 and L4 layers:

> -        *               has_l3, has_l4, has_ipv4, has_ipv6, has_ipfrag,

> -        *               has_ipsec, has_udp, has_tcp, etc depending on

> -        *               the resulted packet format

> +        *  * pktio:     For inbound inline IPSEC processed packets,

> original

> +        *               packet input interface

>          *

> -        * @see odp_packet_l3_offset(), odp_packet_l4_offset(),

> -        *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),

> -        *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()

> +        *  Other meta-data for parse results and error checks depend on

> +        *  configuration (selected parse and error check levels).

>          */

>         odp_packet_t *pkt;

>

> @@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>  /**

>   * Inbound asynchronous IPSEC operation

>   *

> - * This operation does inbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically

> to

> - * odp_ipsec_in(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering

> considerations.

> + * This operation does inbound IPSEC processing in asynchronous mode. It

> + * processes packets otherwise identically to odp_ipsec_in(), but outputs

> all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler

> context

> @@ -928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>   * events for the same SA are enqueued in order, and packet handles (for

> the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for IPSEC

> + * packets for which inline processing is not possible. Packets for the

> same SA

> + * may be processed simultaneously in both modes (initiated by this

> function

> + * and inline operation).

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt)

> @@ -940,10 +1213,10 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> *input);

>  /**

>   * Outbound asynchronous IPSEC operation

>   *

> - * This operation does outbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically

> to

> - * odp_ipsec_out(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering

> considerations.

> + * This operation does outbound IPSEC processing in asynchronous mode. It

> + * processes packets otherwise identically to odp_ipsec_out(), but

> outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler

> context

> @@ -953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> *input);

>   * events for the same SA are enqueued in order, and packet handles (for

> the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for IPSEC

> + * packets for which inline processing is not possible.

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt)

> @@ -963,6 +1239,28 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> *input);

>  int odp_ipsec_out_enq(const odp_ipsec_op_param_t *input);

>

>  /**

> + * Outbound inline IPSEC operation

> + *

> + * This operation does outbound inline IPSEC processing for the packets.

> It's

> + * otherwise identical to odp_ipsec_out_enq(), but outputs all

> successfully

> + * transformed packets to the specified output interface, instead of

> generating

> + * result events for those.

> + *

> + * Inline operation parameters are defined per packet. The array of

> parameters

> + * must have 'op_param.num_pkt' elements and is pointed to by

> 'inline_param'.

> + *

> + * @param         op_param      Operation parameters

> + * @param         inline_param  Outbound inline operation specific

> parameters

> + *

> + * @return Number of packets consumed (0 ... op_param.num_pkt)

> + * @retval <0     On failure

> + *

> + * @see odp_ipsec_out_enq()

> + */

> +int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,

> +                        const odp_ipsec_inline_op_param_t *inline_param);

> +

> +/**

>   * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event

>   *

>   * Copies IPSEC operation results from an event. The event must be of

> diff --git a/include/odp/api/spec/packet_io.h

> b/include/odp/api/spec/packet_io.h

> index cec1f22..8802089 100644

> --- a/include/odp/api/spec/packet_io.h

> +++ b/include/odp/api/spec/packet_io.h

> @@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {

>          * interface capability before enabling the same. */

>         odp_bool_t enable_loop;

>

> +       /** Inbound IPSEC inlined with packet input

> +        *

> +        *  Enable/disable inline inbound IPSEC operation. When enabled

> packet

> +        *  input directs all IPSEC packets automatically to IPSEC inbound

> +        *  processing. IPSEC configuration is done through the IPSEC API.

> +        *  Packets that are not (recognized as) IPSEC are processed

> +        *  according to the packet input configuration.

> +        *

> +        *  0: Disable inbound IPSEC inline operation (default)

> +        *  1: Enable inbound IPSEC inline operation

> +        *

> +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +        */

> +       odp_bool_t inbound_ipsec;

> +

> +       /** Outbound IPSEC inlined with packet output

> +        *

> +        *  Enable/disable inline outbound IPSEC operation. When enabled

> IPSEC

> +        *  outbound processing can send outgoing IPSEC packets directly

> +        *  to the pktio interface for output. IPSEC configuration is done

> +        *  through the IPSEC API.

> +        *

> +        *  Outbound IPSEC inline operation cannot be combined with traffic

> +        *  manager (ODP_PKTOUT_MODE_TM).

>


We need to fix this since TM is likely to be the preferred TX mode, just as
the classifier is the preferred RX mode. There's no reason why IPsec
traffic shouldn't also be subject to TM policies.


> +        *

> +        *  0: Disable outbound IPSEC inline operation (default)

> +        *  1: Enable outbound IPSEC inline operation

> +        *

> +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +        */

> +       odp_bool_t outbound_ipsec;

> +

>  } odp_pktio_config_t;

>

>  /**

> --

> 2.8.1

>

>

Hi,

> > +

> > +       /** Outer headers for inline output operation

> > +        *

> > +        *  Outbound inline IPSEC operation uses this information to

> > prepend

> > +        *  outer headers to the IPSEC packet before sending it out.

> > +        */

> > +       struct {

> > +               /** Points to first byte of outer headers to be copied in

> > +                *  front of the outgoing IPSEC packet. Implementation

> > copies

> > +                *  the headers during odp_ipsec_out_inline() call. */

> > +               uint8_t *ptr;

> >

> 

> Should this be an odp_packet_t rather than a raw set of bytes? The

> rationale for making this an odp_packet_t is that this could then be a

> packet reference which can be used for (re)transmit tracking. For input


If it were odp_packet_t, the IPsec implementation would have to be
prepared for the data to be segmented in arbitrary ways, which probably
would not improve performance. And to me it is not really a packet anyway.

I did not quite understand the point about retransmit tracking in this
context.

One thing we already discussed with Petri is that to enable IP and UDP
based tunneling protocols (e.g. IPIP, GRE, VxLAN) outside the IPsec
packet in the inline output case, the right packet length would have
to be inserted in the right place of the added header. There is the
same issue with L2 protocols that would contain the total packet
length (e.g. some proprietary fabric encapsulation). In this case
the outer header is rather like a template. This is something to be
considered in some future versions of the API but one possible
way to accommodate some of these cases is to have an offset that
specifies which word of the complete packet shall be incremented
by the resulting packet size after IPsec encapsulation.

> having this be raw bytes makes sense since the ODP implementation is

> controlling their allocation and placement, however for output how is the

> application going to allocate these? Presumably it would create a header

> via odp_packet_alloc() and then use odp_packet_data() to get this address,

> but that seems cumbersome compared to simply passing the odp_packet_t

> directly. The alternative would seem to require that the application use

> some sort of odp_shm_reserve() call, but that gets awkward and is far less

> efficient than packet allocation, which is a fastpath operation.

>


I do not think applications would dynamically allocate the storage for the
header (through packet alloc or shm alloc or whatever) for every packet sent
but would just have the pointer point to a cached header somewhere (e.g.
ready-made L2 header cached in the next hop structure found by L3 route
search). If the header or part of it does have to be constructed on the
fly, the application can use the stack or other thread specific storage
that does not have to be allocated separately for every output operation.
Or maybe even some existing per-packet memory like the user area.

Btw, the API should make it clear if the header data can be in the headroom
or the user area of the packet being enqueued (I suppose the latter should
be fine).
 
	Janne


> -----Original Message-----

> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Bill Fischofer

> Sent: Wednesday, March 22, 2017 12:17 AM

> To: Petri Savolainen <petri.savolainen@linaro.org>

> Cc: lng-odp-forward <lng-odp@lists.linaro.org>

> Subject: Re: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

> 

> On Tue, Mar 21, 2017 at 9:17 AM, Petri Savolainen <

> petri.savolainen@linaro.org> wrote:

> 

> > Added support for inline IPSEC processing on packet input and

> > output. Inline mode IPSEC and traffic manager cannot be enabled

> > (currently) on the same pktio interface.

> >

> > Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

> > ---

> >  include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++

> > ++++++---

> >  include/odp/api/spec/packet_io.h |  32 ++++

> >  2 files changed, 355 insertions(+), 25 deletions(-)

> >

> > diff --git a/include/odp/api/spec/ipsec.h b/include/odp/api/spec/ipsec.h

> > index e951e49..23d02cf 100644

> > --- a/include/odp/api/spec/ipsec.h

> > +++ b/include/odp/api/spec/ipsec.h

> > @@ -19,6 +19,8 @@ extern "C" {

> >  #endif

> >

> >  #include <odp/api/crypto.h>

> > +#include <odp/api/packet_io.h>

> > +#include <odp/api/classification.h>

> >

> >  /** @defgroup odp_ipsec ODP IPSEC

> >   *  Operations of IPSEC API.

> > @@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {

> >           * Application uses asynchronous IPSEC operations,

> >           * which return results via events.

> >           */

> > -       ODP_IPSEC_OP_MODE_ASYNC

> > +       ODP_IPSEC_OP_MODE_ASYNC,

> > +

> > +       /** Inline IPSEC operation

> > +         *

> > +         * Packet input/output is connected directly to IPSEC

> > inbound/outbound

> > +         * processing. Application uses asynchronous or inline IPSEC

> > +         * operations.

> > +         */

> > +       ODP_IPSEC_OP_MODE_INLINE,

> > +

> > +       /** IPSEC is disabled in inbound / outbound direction */

> > +       ODP_IPSEC_OP_MODE_DISABLED

> >

> >  } odp_ipsec_op_mode_t;

> >

> >  /**

> > + * Protocol layers in IPSEC configuration

> > + */

> > +typedef enum odp_ipsec_proto_layer_t {

> > +       /** No layers */

> > +       ODP_IPSEC_LAYER_NONE = 0,

> > +

> > +       /** Layer L2 protocols (Ethernet, VLAN, etc) */

> > +       ODP_IPSEC_LAYER_L2,

> > +

> > +       /** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

> >

> 

> Nit: Correct spelling is IPsec, not IPSec, per RFC 4301.

> 

> 

> > +       ODP_IPSEC_LAYER_L3,

> > +

> > +       /** Layer L4 protocols (UDP, TCP, SCTP) */

> > +       ODP_IPSEC_LAYER_L4,

> > +

> > +       /** All layers */

> > +       ODP_IPSEC_LAYER_ALL

> > +

> > +} odp_ipsec_proto_layer_t;

> > +

> > +/**

> >   * Configuration options for IPSEC inbound processing

> >   */

> >  typedef struct odp_ipsec_inbound_config_t {

> > @@ -77,9 +111,113 @@ typedef struct odp_ipsec_inbound_config_t {

> >                 uint32_t max;

> >         } spi;

> >

> > +       /** Retain outer headers

> > +        *

> > +        *  Select up to which protocol layer (at least) outer headers are

> > +        *  retained in inbound inline processing. Default value is

> > +        *  ODP_IPSEC_LAYER_NONE.

> > +        *

> > +        *  ODP_IPSEC_LAYER_NONE: Application does not require any outer

> > +        *                        headers to be retained.

> > +        *

> > +        *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.

> > +        *

> > +        *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3, otherwise

> > the

> > +        *                        same as ODP_IPSEC_LAYER_ALL.

> > +        *

> > +        *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4, otherwise

> > the

> > +        *                        same as ODP_IPSEC_LAYER_ALL.

> > +        *

> > +        *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before IPSEC

> > are

> > +        *                        retained. In transport mode, all headers

> > +        *                        before IP (carrying IPSEC) are retained.

> > +        *

> > +        */

> > +       odp_ipsec_proto_layer_t retain_outer;

> > +

> > +       /** Parse packet headers in IPSEC payload

> > +        *

> > +        *  Select header parsing level after inbound processing. Packet

> > headers

> > +        *  in IPSEC payload must be parsed (at least) up to this level.

> > +        *  Default value is ODP_IPSEC_LAYER_NONE.

> > +        *

> > +        *  Note that IPSec payload is never a L2 packet

> > (ODP_IPSEC_LAYER_L2

> >

> 

> IPsec

> 

> 

> > +        *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC payload

> > +        *  starts after IP header (ODP_IPSEC_LAYER_L3 equals

> > +        *  ODP_IPSEC_LAYER_NONE).

> > +        */

> > +       odp_ipsec_proto_layer_t parse;

> > +

> > +       /** Flags to control IPSEC payload data checks up to the selected

> > parse

> > +        *  level. */

> > +       union {

> > +               struct {

> > +                       /** Check IPv4 header checksum in IPSEC payload.

> > +                        *  Default value is 0. */

> > +                       uint32_t ipv4_chksum   : 1;

> > +

> > +                       /** Check UDP checksum in IPSEC payload.

> > +                        *  Default value is 0. */

> > +                       uint32_t udp_chksum    : 1;

> > +

> > +                       /** Check TCP checksum in IPSEC payload.

> > +                        *  Default value is 0. */

> > +                       uint32_t tcp_chksum    : 1;

> > +

> > +                       /** Check SCTP checksum in IPSEC payload.

> > +                        *  Default value is 0. */

> > +                       uint32_t sctp_chksum   : 1;

> > +               } check;

> > +

> > +               /** All bits of the bit field structure

> > +                 *

> > +                 * This field can be used to set/clear all flags, or

> > bitwise

> > +                 * operations over the entire structure. */

> > +               uint32_t all_check;

> > +       };

> > +

> >  } odp_ipsec_inbound_config_t;

> >

> >  /**

> > + * Configuration options for IPSEC outbound processing

> > + */

> > +typedef struct odp_ipsec_outbound_config_t {

> > +       /** Flags to control L3/L4 checksum insertion as part of outbound

> > +        *  packet processing. Packet must have set with valid L3/L4

> > offsets.

> > +        *  Checksum configuration is ignored for packets that checksum

> > cannot

> > +        *  be computed for (e.g. IPv4 fragments). Application may use a

> > packet

> > +        *  metadata flag to disable checksum insertion per packet bases.

> > +        */

> > +       union {

> > +               struct {

> > +                       /** Insert IPv4 header checksum on the payload

> > packet

> > +                        *  before IPSEC transformation. Default value is

> > 0. */

> > +                       uint32_t inner_ipv4   : 1;

> > +

> > +                       /** Insert UDP header checksum on the payload

> > packet

> > +                        *  before IPSEC transformation. Default value is

> > 0. */

> > +                       uint32_t inner_udp    : 1;

> > +

> > +                       /** Insert TCP header checksum on the payload

> > packet

> > +                        *  before IPSEC transformation. Default value is

> > 0. */

> > +                       uint32_t inner_tcp    : 1;

> > +

> > +                       /** Insert SCTP header checksum on the payload

> > packet

> > +                        *  before IPSEC transformation. Default value is

> > 0. */

> > +                       uint32_t inner_sctp   : 1;

> > +

> > +               } chksum;

> > +

> > +               /** All bits of the bit field structure

> > +                 *

> > +                 * This field can be used to set/clear all flags, or

> > bitwise

> > +                 * operations over the entire structure. */

> > +               uint32_t all_chksum;

> > +       };

> > +

> > +} odp_ipsec_outbound_config_t;

> > +

> > +/**

> >   * IPSEC capability

> >   */

> >  typedef struct odp_ipsec_capability_t {

> > @@ -102,6 +240,24 @@ typedef struct odp_ipsec_capability_t {

> >          */

> >         uint8_t op_mode_async;

> >

> > +       /** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE) support

> > +        *

> > +        *  0: Inline IPSEC operation is not supported

> > +        *  1: Inline IPSEC operation is supported

> > +        *  2: Inline IPSEC operation is supported and preferred

> > +        */

> > +       uint8_t op_mode_inline;

> > +

> > +       /** Support of inline classification (ODP_IPSEC_DEST_CLS) for

> > resulting

> > +        *  inbound packets.

> > +        *

> > +        *  0: Inline classification of resulting packets is not supported

> > +        *  1: Inline classification of resulting packets is supported

> > +        *  2: Inline classification of resulting packets is supported and

> > +        *     preferred

> > +        */

> > +       uint8_t cls_inline;

> > +

> >         /** Soft expiry limit in seconds support

> >          *

> >          *  0: Limit is not supported

> > @@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {

> >   * IPSEC configuration options

> >   */

> >  typedef struct odp_ipsec_config_t {

> > -       /** IPSEC operation mode. Application selects which mode (sync or

> > async)

> > -        *  will be used for IPSEC operations.

> > +       /** Inbound IPSEC operation mode. Application selects which mode

> > +        *  will be used for inbound IPSEC operations.

> >          *

> >          *  @see odp_ipsec_in(), odp_ipsec_in_enq()

> >          */

> > -       odp_ipsec_op_mode_t op_mode;

> > +       odp_ipsec_op_mode_t inbound_mode;

> > +

> > +       /** Outbound IPSEC operation mode. Application selects which mode

> > +        *  will be used for outbound IPSEC operations.

> > +        *

> > +        *  @see odp_ipsec_out(), odp_ipsec_out_enq(),

> > odp_ipsec_out_inline()

> > +        */

> > +       odp_ipsec_op_mode_t outbound_mode;

> >

> >         /** Maximum number of IPSEC SAs that application will use

> >          * simultaneously */

> > @@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {

> >         /** IPSEC inbound processing configuration */

> >         odp_ipsec_inbound_config_t inbound;

> >

> > +       /** IPSEC outbound processing configuration */

> > +       odp_ipsec_outbound_config_t outbound;

> > +

> >  } odp_ipsec_config_t;

> >

> >  /**

> > @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

> >         ODP_IPSEC_LOOKUP_DISABLED = 0,

> >

> >         /** Inbound SA lookup is enabled. Used SPI values must be unique.

> > */

> > -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> > +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> > +

> > +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> > +         * destination IP address. Used SPI values must be unique. */

> > +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

> >

> >  } odp_ipsec_lookup_mode_t;

> >

> >  /**

> > + * Result event destination

> > + */

> > +typedef enum odp_ipsec_dest_mode_t {

> > +       /** Destination for IPSEC result events is a queue. */

> > +       ODP_IPSEC_DEST_QUEUE = 0,

> > +

> > +       /** Destination for IPSEC result events is the classifier.

> > +        *  IPSEC capability 'cls_inline' determines if inline

> > classification

> > +        *  is supported. */

> > +       ODP_IPSEC_DEST_CLS

> > +

> > +} odp_ipsec_dest_mode_t;

> > +

> > +/**

> >   * IPSEC Security Association (SA) parameters

> >   */

> >  typedef struct odp_ipsec_sa_param_t {

> > @@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {

> >         /** SPI value */

> >         uint32_t spi;

> >

> > +       /** Additional inbound SA lookup parameters. Values are considered

> > +        *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */

> > +       struct {

> > +               /* v4 or v6 */

> > +               uint8_t ip_version;

> >

> 

> Is this an enum? If not perhaps doc this as 4 = IPv4, 6 = IPv6.

> 

> 

> > +

> > +               /* IP destination address (NETWORK ENDIAN) */

> > +               void    *dst_addr;

> > +

> > +       } lookup_param;

> > +

> >         /** MTU for outbound IP fragmentation offload

> >          *

> >          *  This is the maximum length of IP packets that outbound IPSEC

> > @@ -434,13 +629,32 @@ typedef struct odp_ipsec_sa_param_t {

> >          */

> >         uint32_t mtu;

> >

> > +       /** Select where IPSEC result events are sent

> > +        *

> > +        *  Asynchronous and inline modes generate result events. Select

> > where

> > +        *  those events are sent. Inbound SAs may choose between a queue

> > or

> > +        *  the classifier. Outbound SAs must define a queue always.

> > +        *  The default value is ODP_IPSEC_DEST_QUEUE.

> > +        */

> > +       odp_ipsec_dest_mode_t dest_mode;

> > +

> >         /** Destination queue for IPSEC events

> >          *

> > -        *  Operations in asynchronous mode enqueue resulting events into

> > -        *  this queue.

> > +        *  Operations in asynchronous or inline mode enqueue resulting

> > events

> > +        *  into this queue.

> >          */

> >         odp_queue_t dest_queue;

> >

> > +       /** Classifier destination CoS for IPSEC result events

> > +        *

> > +        *  Result events for successfully decapsulated packets are sent to

> > +        *  classification through this CoS. Other result events are sent

> > to

> > +        *  'dest_queue'. This field is considered only when 'dest_mode' is

> > +        *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any

> > pktio

> > +        *  interface default CoS.

> > +        */

> > +       odp_cos_t dest_cos;

> > +

> >         /** User defined SA context pointer

> >          *

> >          *  User defined context pointer associated with the SA.

> > @@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {

> >                 uint32_t all_error;

> >         };

> >

> > +       union {

> > +               /** Status flags */

> > +               struct {

> > +                       /** Packet was processed in inline mode */

> > +                       uint32_t inline_mode      : 1;

> > +

> > +               } flag;

> > +

> > +               /** All flag bits */

> > +               uint32_t all_flag;

> > +       };

> > +

> >  } odp_ipsec_op_status_t;

> >

> >  /**

> > @@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {

> >  } odp_ipsec_op_param_t;

> >

> >  /**

> > + * Outbound inline IPSEC operation parameters

> > + */

> > +typedef struct odp_ipsec_inline_op_param_t {

> > +       /** Packet output interface for inline output operation

> > +        *

> > +        *  Outbound inline IPSEC operation uses this packet IO interface

> > to

> > +        *  output the packet after a successful IPSEC transformation. The

> > pktio

> > +        *  must have been configured to operate in inline IPSEC mode.

> > +        */

> > +       odp_pktio_t pktio;

> >

> 

> Shouldn't we have an option for this to be output to an odp_tm_queue_t

> instead of a pktio?

> Also, for output are there any controls over which odp_pktout_queue_t is

> used, or is this undefined?

> 

> 

> > +

> > +       /** Outer headers for inline output operation

> > +        *

> > +        *  Outbound inline IPSEC operation uses this information to

> > prepend

> > +        *  outer headers to the IPSEC packet before sending it out.

> > +        */

> > +       struct {

> > +               /** Points to first byte of outer headers to be copied in

> > +                *  front of the outgoing IPSEC packet. Implementation

> > copies

> > +                *  the headers during odp_ipsec_out_inline() call. */

> > +               uint8_t *ptr;

> >

> 

> Should this be an odp_packet_t rather than a raw set of bytes? The

> rationale for making this an odp_packet_t is that this could then be a

> packet reference which can be used for (re)transmit tracking. For input

> having this be raw bytes makes sense since the ODP implementation is

> controlling their allocation and placement, however for output how is the

> application going to allocate these? Presumably it would create a header

> via odp_packet_alloc() and then use odp_packet_data() to get this address,

> but that seems cumbersome compared to simply passing the odp_packet_t

> directly. The alternative would seem to require that the application use

> some sort of odp_shm_reserve() call, but that gets awkward and is far less

> efficient than packet allocation, which is a fastpath operation.

> 

> 

> > +

> > +               /** Outer header length in bytes */

> > +               uint32_t len;

> > +       } outer_hdr;

> > +

> > +} odp_ipsec_inline_op_param_t;

> > +

> > +/**

> >   * IPSEC operation result for a packet

> >   */

> >  typedef struct odp_ipsec_packet_result_t {

> > @@ -752,6 +1007,23 @@ typedef struct odp_ipsec_packet_result_t {

> >          */

> >         odp_ipsec_sa_t sa;

> >

> > +       /** Packet outer header status before inbound inline processing.

> > +        *  This is valid only when status.flag.inline_mode is set.

> > +        */

> > +       struct {

> > +               /** Points to the first byte of retained outer headers.

> > These

> > +                *  headers are stored in a contiquous, per packet,

> > +                *  implementation specific memory space. Since the memory

> > space

> > +                *  may overlap with e.g. packet head/tailroom, the content

> > +                *  becomes invalid if packet data storage is modified in

> > +                *  anyway. The memory space may not be sharable to other

> > +                *  threads. */

> > +               uint8_t *ptr;

> > +

> > +               /** Outer header length in bytes */

> > +               uint32_t len;

> > +       } outer_hdr;

> > +

> >  } odp_ipsec_packet_result_t;

> >

> >  /**

> > @@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {

> >          *  at least 'num_pkt' elements.

> >          *

> >          *  Each successfully transformed packet has a valid value for

> > these

> > -        *  meta-data:

> > +        *  meta-data regardless of the inner packet parse configuration.

> >

> 

> Vestigial typo. It's metadata, not meta-data. Might as well clean up here.

> 

> 

> > +        *  (odp_ipsec_inbound_config_t):

> >          *  * L3 offset: Offset to the first byte of the (outmost) IP

> > header

> > -        *  * L4 offset: Offset to the first byte of the valid and known L4

> > -        *               header (immediately following the IP header).

> > -        *  * Various flags about L3 and L4 layers:

> > -        *               has_l3, has_l4, has_ipv4, has_ipv6, has_ipfrag,

> > -        *               has_ipsec, has_udp, has_tcp, etc depending on

> > -        *               the resulted packet format

> > +        *  * pktio:     For inbound inline IPSEC processed packets,

> > original

> > +        *               packet input interface

> >          *

> > -        * @see odp_packet_l3_offset(), odp_packet_l4_offset(),

> > -        *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),

> > -        *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()

> > +        *  Other meta-data for parse results and error checks depend on

> > +        *  configuration (selected parse and error check levels).

> >          */

> >         odp_packet_t *pkt;

> >

> > @@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

> >  /**

> >   * Inbound asynchronous IPSEC operation

> >   *

> > - * This operation does inbound IPSEC processing in asynchronous mode

> > - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically

> > to

> > - * odp_ipsec_in(), but outputs all results through one or more

> > - * ODP_EVENT_IPSEC_RESULT events with the following ordering

> > considerations.

> > + * This operation does inbound IPSEC processing in asynchronous mode. It

> > + * processes packets otherwise identically to odp_ipsec_in(), but outputs

> > all

> > + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> > following

> > + * ordering considerations.

> >   *

> >   * Asynchronous mode maintains (operation input) packet order per SA when

> >   * application calls the operation within an ordered or atomic scheduler

> > context

> > @@ -928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

> >   * events for the same SA are enqueued in order, and packet handles (for

> > the

> >   * same SA) are stored in order within an event.

> >   *

> > + * The function may be used also in inline processing mode, e.g. for IPSEC

> > + * packets for which inline processing is not possible. Packets for the

> > same SA

> > + * may be processed simultaneously in both modes (initiated by this

> > function

> > + * and inline operation).

> > + *

> >   * @param         input   Operation input parameters

> >   *

> >   * @return Number of input packets consumed (0 ... input.num_pkt)

> > @@ -940,10 +1213,10 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> > *input);

> >  /**

> >   * Outbound asynchronous IPSEC operation

> >   *

> > - * This operation does outbound IPSEC processing in asynchronous mode

> > - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically

> > to

> > - * odp_ipsec_out(), but outputs all results through one or more

> > - * ODP_EVENT_IPSEC_RESULT events with the following ordering

> > considerations.

> > + * This operation does outbound IPSEC processing in asynchronous mode. It

> > + * processes packets otherwise identically to odp_ipsec_out(), but

> > outputs all

> > + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> > following

> > + * ordering considerations.

> >   *

> >   * Asynchronous mode maintains (operation input) packet order per SA when

> >   * application calls the operation within an ordered or atomic scheduler

> > context

> > @@ -953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> > *input);

> >   * events for the same SA are enqueued in order, and packet handles (for

> > the

> >   * same SA) are stored in order within an event.

> >   *

> > + * The function may be used also in inline processing mode, e.g. for IPSEC

> > + * packets for which inline processing is not possible.

> > + *

> >   * @param         input   Operation input parameters

> >   *

> >   * @return Number of input packets consumed (0 ... input.num_pkt)

> > @@ -963,6 +1239,28 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> > *input);

> >  int odp_ipsec_out_enq(const odp_ipsec_op_param_t *input);

> >

> >  /**

> > + * Outbound inline IPSEC operation

> > + *

> > + * This operation does outbound inline IPSEC processing for the packets.

> > It's

> > + * otherwise identical to odp_ipsec_out_enq(), but outputs all

> > successfully

> > + * transformed packets to the specified output interface, instead of

> > generating

> > + * result events for those.

> > + *

> > + * Inline operation parameters are defined per packet. The array of

> > parameters

> > + * must have 'op_param.num_pkt' elements and is pointed to by

> > 'inline_param'.

> > + *

> > + * @param         op_param      Operation parameters

> > + * @param         inline_param  Outbound inline operation specific

> > parameters

> > + *

> > + * @return Number of packets consumed (0 ... op_param.num_pkt)

> > + * @retval <0     On failure

> > + *

> > + * @see odp_ipsec_out_enq()

> > + */

> > +int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,

> > +                        const odp_ipsec_inline_op_param_t *inline_param);

> > +

> > +/**

> >   * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event

> >   *

> >   * Copies IPSEC operation results from an event. The event must be of

> > diff --git a/include/odp/api/spec/packet_io.h

> > b/include/odp/api/spec/packet_io.h

> > index cec1f22..8802089 100644

> > --- a/include/odp/api/spec/packet_io.h

> > +++ b/include/odp/api/spec/packet_io.h

> > @@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {

> >          * interface capability before enabling the same. */

> >         odp_bool_t enable_loop;

> >

> > +       /** Inbound IPSEC inlined with packet input

> > +        *

> > +        *  Enable/disable inline inbound IPSEC operation. When enabled

> > packet

> > +        *  input directs all IPSEC packets automatically to IPSEC inbound

> > +        *  processing. IPSEC configuration is done through the IPSEC API.

> > +        *  Packets that are not (recognized as) IPSEC are processed

> > +        *  according to the packet input configuration.

> > +        *

> > +        *  0: Disable inbound IPSEC inline operation (default)

> > +        *  1: Enable inbound IPSEC inline operation

> > +        *

> > +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> > +        */

> > +       odp_bool_t inbound_ipsec;

> > +

> > +       /** Outbound IPSEC inlined with packet output

> > +        *

> > +        *  Enable/disable inline outbound IPSEC operation. When enabled

> > IPSEC

> > +        *  outbound processing can send outgoing IPSEC packets directly

> > +        *  to the pktio interface for output. IPSEC configuration is done

> > +        *  through the IPSEC API.

> > +        *

> > +        *  Outbound IPSEC inline operation cannot be combined with traffic

> > +        *  manager (ODP_PKTOUT_MODE_TM).

> >

> 

> We need to fix this since TM is likely to be the preferred TX mode, just as

> the classifier is the preferred RX mode. There's no reason why IPsec

> traffic shouldn't also be subject to TM policies.

> 

> 

> > +        *

> > +        *  0: Disable outbound IPSEC inline operation (default)

> > +        *  1: Enable outbound IPSEC inline operation

> > +        *

> > +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> > +        */

> > +       odp_bool_t outbound_ipsec;

> > +

> >  } odp_pktio_config_t;

> >

> >  /**

> > --

> > 2.8.1

> >

> >

NO HTML mails to the ODP list please. Indentation is lost due to HTML.


 /**
+ * Protocol layers in IPSEC configuration
+ */
+typedef enum odp_ipsec_proto_layer_t {
+       /** No layers */
+       ODP_IPSEC_LAYER_NONE = 0,
+
+       /** Layer L2 protocols (Ethernet, VLAN, etc) */
+       ODP_IPSEC_LAYER_L2,
+
+       /** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

Nit: Correct spelling is IPsec, not IPSec, per RFC 4301.

+       /** Parse packet headers in IPSEC payload
+        *
+        *  Select header parsing level after inbound processing. Packet headers
+        *  in IPSEC payload must be parsed (at least) up to this level.
+        *  Default value is ODP_IPSEC_LAYER_NONE.
+        *
+        *  Note that IPSec payload is never a L2 packet (ODP_IPSEC_LAYER_L2

IPsec


OK. I'll correct it to IPSEC, which is used throughout this file.

" All other capitalizations of IPsec (e.g., IPSEC, IPSec, ipsec) are deprecated. However, any capitalization of the  sequence of letters "IPsec" should be understood to refer to the IPsec protocols. "

 
+/**
  * IPSEC Security Association (SA) parameters
  */
 typedef struct odp_ipsec_sa_param_t {
@@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {
        /** SPI value */
        uint32_t spi;

+       /** Additional inbound SA lookup parameters. Values are considered
+        *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */
+       struct {
+               /* v4 or v6 */
+               uint8_t ip_version;

Is this an enum? If not perhaps doc this as 4 = IPv4, 6 = IPv6.

OK. I'll document it.

 

 /**
+ * Outbound inline IPSEC operation parameters
+ */
+typedef struct odp_ipsec_inline_op_param_t {
+       /** Packet output interface for inline output operation
+        *
+        *  Outbound inline IPSEC operation uses this packet IO interface to
+        *  output the packet after a successful IPSEC transformation. The pktio
+        *  must have been configured to operate in inline IPSEC mode.
+        */
+       odp_pktio_t pktio;

Shouldn't we have an option for this to be output to an odp_tm_queue_t instead of a pktio? 
Also, for output are there any controls over which odp_pktout_queue_t is used, or is this undefined?


TM is not supported at this stage. Destination TM queue will be added here when TM support is added.

Pktout queues are for application use, there's no benefit from those to an accelerator (IPSEC may use whatever  implementation specific way to deliver packet to pktio).

 
+
+       /** Outer headers for inline output operation
+        *
+        *  Outbound inline IPSEC operation uses this information to prepend
+        *  outer headers to the IPSEC packet before sending it out.
+        */
+       struct {
+               /** Points to first byte of outer headers to be copied in
+                *  front of the outgoing IPSEC packet. Implementation copies
+                *  the headers during odp_ipsec_out_inline() call. */
+               uint8_t *ptr;

Should this be an odp_packet_t rather than a raw set of bytes? The rationale for making this an odp_packet_t is that this could then be a packet reference which can be used for (re)transmit tracking. For input having this be raw bytes makes sense since the ODP implementation is controlling their allocation and placement, however for output how is the application going to allocate these? Presumably it would create a header via odp_packet_alloc() and then use odp_packet_data() to get this address, but that seems cumbersome compared to simply passing the odp_packet_t directly. The alternative would seem to require that the application use some sort of odp_shm_reserve() call, but that gets awkward and is far less efficient than packet allocation, which is a fastpath operation.
 

Packet is not needed for l2 hdr storage. Application will store headers to e.g. SA context, etc convenient memory location and pass only a pointer to there. Implementation copies those (few) bytes during odp_ipsec_out_inline() call. Since headers are copied, there's no special requirement for the memory - also stack can be used. Most likely application would not store the headers into a packet - since there's no benefit on doing so. The payload packet would be more likely target for reference counting (than l2 hdr bytes).


 /**
@@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {
         *  at least 'num_pkt' elements.
         *
         *  Each successfully transformed packet has a valid value for these
-        *  meta-data:
+        *  meta-data regardless of the inner packet parse configuration.

Vestigial typo. It's metadata, not meta-data. Might as well clean up here.

OK.

+
+       /** Outbound IPSEC inlined with packet output
+        *
+        *  Enable/disable inline outbound IPSEC operation. When enabled IPSEC
+        *  outbound processing can send outgoing IPSEC packets directly
+        *  to the pktio interface for output. IPSEC configuration is done
+        *  through the IPSEC API.
+        *
+        *  Outbound IPSEC inline operation cannot be combined with traffic
+        *  manager (ODP_PKTOUT_MODE_TM).

We need to fix this since TM is likely to be the preferred TX mode, just as the classifier is the preferred RX mode. There's no reason why IPsec traffic shouldn't also be subject to TM policies. 


Me, Bala and Janne think that it's better to leave TM integration as a next step. Classifier integration is much more important, since it enables load balancing even for a single, fat IPSEC tunnel. TM is an additional feature that may, or may not be used (IPSEC does not need it). Application may still shape traffic in SW before calling IPSEC output.


-Petri

On Wed, Mar 22, 2017 at 6:37 AM, Peltonen, Janne (Nokia - FI/Espoo) <
janne.peltonen@nokia.com> wrote:

> Hi,

>

> > > +

> > > +       /** Outer headers for inline output operation

> > > +        *

> > > +        *  Outbound inline IPSEC operation uses this information to

> > > prepend

> > > +        *  outer headers to the IPSEC packet before sending it out.

> > > +        */

> > > +       struct {

> > > +               /** Points to first byte of outer headers to be copied

> in

> > > +                *  front of the outgoing IPSEC packet. Implementation

> > > copies

> > > +                *  the headers during odp_ipsec_out_inline() call. */

> > > +               uint8_t *ptr;

> > >

> >

> > Should this be an odp_packet_t rather than a raw set of bytes? The

> > rationale for making this an odp_packet_t is that this could then be a

> > packet reference which can be used for (re)transmit tracking. For input

>

> If it were odp_packet_t, the IPsec implementation would have to be

> prepared for the data to be segmented in arbitrary ways, which probably

> would not improve performance. And to me it is not really a packet anyway.

>


It's as much of a packet as the hdr passed to odp_packet_ref_pkt() to be
prefixed to the created reference. An odp_packet_t is how we represent
packets (complete and partial) in ODP, so it makes sense to be consistent
here. We already have minimum segment sizes in the packet definition so I
don't see this being a concern here any more than in any other ODP packet
APIs.


>

> I did not quite understand the point about retransmit tracking in this

> context.

>


One of the main use cases for references is to permit easy retransmit
processing via constructs like:

odp_pktout_send(outq, odp_packet_ref_static(pkt));

That allows the caller to hold onto a reference to the packet being
transmitted both for retransmit purposes as well as to be able to determine
that that transmit has been successful, indicated by
odp_packet_has_ref(pkt) returning from 1 to 0. You can't really pass a
reference to the IPsec routines since references have read-only payloads
and IPsec is transforming the payload, however if you have a separate
header that can be a reference since all you're asking is that this be
prefixed to the result prior to transmission, so it would seem the same
capabilities would result.


>

> One thing we already discussed with Petri is that to enable IP and UDP

> based tunneling protocols (e.g. IPIP, GRE, VxLAN) outside the IPsec

> packet in the inline output case, the right packet length would have

> to be inserted in the right place of the added header. There is the

> same issue with L2 protocols that would contain the total packet

> length (e.g. some proprietary fabric encapsulation). In this case

> the outer header is rather like a template. This is something to be

> considered in some future versions of the API but one possible

> way to accommodate some of these cases is to have an offset that

> specifies which word of the complete packet shall be incremented

> by the resulting packet size after IPsec encapsulation.

>


I agree this is a direction we want to go, but I'd want to see this done
via a generalized tunnel acceleration set of APIs the permit general
encapsulation/decapsulation services. Given the flexibility inherent in
this sort of processing, I see this as one area where synergy with the P4
project is most promising. P4 excels at offering highly efficient
descriptions of bit-level packet structures and manipulations that can be
implemented in soft logic like FPGAs. The idea would be that the
application would either select from a library of predefined P4 templates
or be able to specify its own unique formats without requiring a new ODP
API for every such variant. Something worth exploring.


>

> > having this be raw bytes makes sense since the ODP implementation is

> > controlling their allocation and placement, however for output how is the

> > application going to allocate these? Presumably it would create a header

> > via odp_packet_alloc() and then use odp_packet_data() to get this

> address,

> > but that seems cumbersome compared to simply passing the odp_packet_t

> > directly. The alternative would seem to require that the application use

> > some sort of odp_shm_reserve() call, but that gets awkward and is far

> less

> > efficient than packet allocation, which is a fastpath operation.

> >

>

> I do not think applications would dynamically allocate the storage for the

> header (through packet alloc or shm alloc or whatever) for every packet

> sent

> but would just have the pointer point to a cached header somewhere (e.g.

> ready-made L2 header cached in the next hop structure found by L3 route

> search). If the header or part of it does have to be constructed on the

> fly, the application can use the stack or other thread specific storage

> that does not have to be allocated separately for every output operation.

> Or maybe even some existing per-packet memory like the user area.

>


The user area would be safe, however the stack would not unless you're
using purely synchronous calls, which is a limit I'd think we wouldn't want
to impose on this feature. The user area, however, would still need
per-packet initialization. The reason for leveraging the packet structure
would be to permit references to these headers to be used as a cheap
cloning mechanism that would leverage whatever HW packet manager structures
are in place to support other packets.


>

> Btw, the API should make it clear if the header data can be in the headroom

> or the user area of the packet being enqueued (I suppose the latter should

> be fine).

>


It's dangerous to try to use headroom since that's essentially trying to
make use of unallocated space. As far as the application is concerned, this
space is unusable (or even non-existent) until after a successful
odp_packet_push_head() call is made.


>

>         Janne

>

>

> > -----Original Message-----

> > From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of

> Bill Fischofer

> > Sent: Wednesday, March 22, 2017 12:17 AM

> > To: Petri Savolainen <petri.savolainen@linaro.org>

> > Cc: lng-odp-forward <lng-odp@lists.linaro.org>

> > Subject: Re: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline

> IPSEC support

> >

> > On Tue, Mar 21, 2017 at 9:17 AM, Petri Savolainen <

> > petri.savolainen@linaro.org> wrote:

> >

> > > Added support for inline IPSEC processing on packet input and

> > > output. Inline mode IPSEC and traffic manager cannot be enabled

> > > (currently) on the same pktio interface.

> > >

> > > Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

> > > ---

> > >  include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++

> > > ++++++---

> > >  include/odp/api/spec/packet_io.h |  32 ++++

> > >  2 files changed, 355 insertions(+), 25 deletions(-)

> > >

> > > diff --git a/include/odp/api/spec/ipsec.h

> b/include/odp/api/spec/ipsec.h

> > > index e951e49..23d02cf 100644

> > > --- a/include/odp/api/spec/ipsec.h

> > > +++ b/include/odp/api/spec/ipsec.h

> > > @@ -19,6 +19,8 @@ extern "C" {

> > >  #endif

> > >

> > >  #include <odp/api/crypto.h>

> > > +#include <odp/api/packet_io.h>

> > > +#include <odp/api/classification.h>

> > >

> > >  /** @defgroup odp_ipsec ODP IPSEC

> > >   *  Operations of IPSEC API.

> > > @@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {

> > >           * Application uses asynchronous IPSEC operations,

> > >           * which return results via events.

> > >           */

> > > -       ODP_IPSEC_OP_MODE_ASYNC

> > > +       ODP_IPSEC_OP_MODE_ASYNC,

> > > +

> > > +       /** Inline IPSEC operation

> > > +         *

> > > +         * Packet input/output is connected directly to IPSEC

> > > inbound/outbound

> > > +         * processing. Application uses asynchronous or inline IPSEC

> > > +         * operations.

> > > +         */

> > > +       ODP_IPSEC_OP_MODE_INLINE,

> > > +

> > > +       /** IPSEC is disabled in inbound / outbound direction */

> > > +       ODP_IPSEC_OP_MODE_DISABLED

> > >

> > >  } odp_ipsec_op_mode_t;

> > >

> > >  /**

> > > + * Protocol layers in IPSEC configuration

> > > + */

> > > +typedef enum odp_ipsec_proto_layer_t {

> > > +       /** No layers */

> > > +       ODP_IPSEC_LAYER_NONE = 0,

> > > +

> > > +       /** Layer L2 protocols (Ethernet, VLAN, etc) */

> > > +       ODP_IPSEC_LAYER_L2,

> > > +

> > > +       /** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

> > >

> >

> > Nit: Correct spelling is IPsec, not IPSec, per RFC 4301.

> >

> >

> > > +       ODP_IPSEC_LAYER_L3,

> > > +

> > > +       /** Layer L4 protocols (UDP, TCP, SCTP) */

> > > +       ODP_IPSEC_LAYER_L4,

> > > +

> > > +       /** All layers */

> > > +       ODP_IPSEC_LAYER_ALL

> > > +

> > > +} odp_ipsec_proto_layer_t;

> > > +

> > > +/**

> > >   * Configuration options for IPSEC inbound processing

> > >   */

> > >  typedef struct odp_ipsec_inbound_config_t {

> > > @@ -77,9 +111,113 @@ typedef struct odp_ipsec_inbound_config_t {

> > >                 uint32_t max;

> > >         } spi;

> > >

> > > +       /** Retain outer headers

> > > +        *

> > > +        *  Select up to which protocol layer (at least) outer headers

> are

> > > +        *  retained in inbound inline processing. Default value is

> > > +        *  ODP_IPSEC_LAYER_NONE.

> > > +        *

> > > +        *  ODP_IPSEC_LAYER_NONE: Application does not require any

> outer

> > > +        *                        headers to be retained.

> > > +        *

> > > +        *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.

> > > +        *

> > > +        *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3,

> otherwise

> > > the

> > > +        *                        same as ODP_IPSEC_LAYER_ALL.

> > > +        *

> > > +        *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4,

> otherwise

> > > the

> > > +        *                        same as ODP_IPSEC_LAYER_ALL.

> > > +        *

> > > +        *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before

> IPSEC

> > > are

> > > +        *                        retained. In transport mode, all

> headers

> > > +        *                        before IP (carrying IPSEC) are

> retained.

> > > +        *

> > > +        */

> > > +       odp_ipsec_proto_layer_t retain_outer;

> > > +

> > > +       /** Parse packet headers in IPSEC payload

> > > +        *

> > > +        *  Select header parsing level after inbound processing.

> Packet

> > > headers

> > > +        *  in IPSEC payload must be parsed (at least) up to this

> level.

> > > +        *  Default value is ODP_IPSEC_LAYER_NONE.

> > > +        *

> > > +        *  Note that IPSec payload is never a L2 packet

> > > (ODP_IPSEC_LAYER_L2

> > >

> >

> > IPsec

> >

> >

> > > +        *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC

> payload

> > > +        *  starts after IP header (ODP_IPSEC_LAYER_L3 equals

> > > +        *  ODP_IPSEC_LAYER_NONE).

> > > +        */

> > > +       odp_ipsec_proto_layer_t parse;

> > > +

> > > +       /** Flags to control IPSEC payload data checks up to the

> selected

> > > parse

> > > +        *  level. */

> > > +       union {

> > > +               struct {

> > > +                       /** Check IPv4 header checksum in IPSEC

> payload.

> > > +                        *  Default value is 0. */

> > > +                       uint32_t ipv4_chksum   : 1;

> > > +

> > > +                       /** Check UDP checksum in IPSEC payload.

> > > +                        *  Default value is 0. */

> > > +                       uint32_t udp_chksum    : 1;

> > > +

> > > +                       /** Check TCP checksum in IPSEC payload.

> > > +                        *  Default value is 0. */

> > > +                       uint32_t tcp_chksum    : 1;

> > > +

> > > +                       /** Check SCTP checksum in IPSEC payload.

> > > +                        *  Default value is 0. */

> > > +                       uint32_t sctp_chksum   : 1;

> > > +               } check;

> > > +

> > > +               /** All bits of the bit field structure

> > > +                 *

> > > +                 * This field can be used to set/clear all flags, or

> > > bitwise

> > > +                 * operations over the entire structure. */

> > > +               uint32_t all_check;

> > > +       };

> > > +

> > >  } odp_ipsec_inbound_config_t;

> > >

> > >  /**

> > > + * Configuration options for IPSEC outbound processing

> > > + */

> > > +typedef struct odp_ipsec_outbound_config_t {

> > > +       /** Flags to control L3/L4 checksum insertion as part of

> outbound

> > > +        *  packet processing. Packet must have set with valid L3/L4

> > > offsets.

> > > +        *  Checksum configuration is ignored for packets that checksum

> > > cannot

> > > +        *  be computed for (e.g. IPv4 fragments). Application may use

> a

> > > packet

> > > +        *  metadata flag to disable checksum insertion per packet

> bases.

> > > +        */

> > > +       union {

> > > +               struct {

> > > +                       /** Insert IPv4 header checksum on the payload

> > > packet

> > > +                        *  before IPSEC transformation. Default value

> is

> > > 0. */

> > > +                       uint32_t inner_ipv4   : 1;

> > > +

> > > +                       /** Insert UDP header checksum on the payload

> > > packet

> > > +                        *  before IPSEC transformation. Default value

> is

> > > 0. */

> > > +                       uint32_t inner_udp    : 1;

> > > +

> > > +                       /** Insert TCP header checksum on the payload

> > > packet

> > > +                        *  before IPSEC transformation. Default value

> is

> > > 0. */

> > > +                       uint32_t inner_tcp    : 1;

> > > +

> > > +                       /** Insert SCTP header checksum on the payload

> > > packet

> > > +                        *  before IPSEC transformation. Default value

> is

> > > 0. */

> > > +                       uint32_t inner_sctp   : 1;

> > > +

> > > +               } chksum;

> > > +

> > > +               /** All bits of the bit field structure

> > > +                 *

> > > +                 * This field can be used to set/clear all flags, or

> > > bitwise

> > > +                 * operations over the entire structure. */

> > > +               uint32_t all_chksum;

> > > +       };

> > > +

> > > +} odp_ipsec_outbound_config_t;

> > > +

> > > +/**

> > >   * IPSEC capability

> > >   */

> > >  typedef struct odp_ipsec_capability_t {

> > > @@ -102,6 +240,24 @@ typedef struct odp_ipsec_capability_t {

> > >          */

> > >         uint8_t op_mode_async;

> > >

> > > +       /** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE)

> support

> > > +        *

> > > +        *  0: Inline IPSEC operation is not supported

> > > +        *  1: Inline IPSEC operation is supported

> > > +        *  2: Inline IPSEC operation is supported and preferred

> > > +        */

> > > +       uint8_t op_mode_inline;

> > > +

> > > +       /** Support of inline classification (ODP_IPSEC_DEST_CLS) for

> > > resulting

> > > +        *  inbound packets.

> > > +        *

> > > +        *  0: Inline classification of resulting packets is not

> supported

> > > +        *  1: Inline classification of resulting packets is supported

> > > +        *  2: Inline classification of resulting packets is supported

> and

> > > +        *     preferred

> > > +        */

> > > +       uint8_t cls_inline;

> > > +

> > >         /** Soft expiry limit in seconds support

> > >          *

> > >          *  0: Limit is not supported

> > > @@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {

> > >   * IPSEC configuration options

> > >   */

> > >  typedef struct odp_ipsec_config_t {

> > > -       /** IPSEC operation mode. Application selects which mode (sync

> or

> > > async)

> > > -        *  will be used for IPSEC operations.

> > > +       /** Inbound IPSEC operation mode. Application selects which

> mode

> > > +        *  will be used for inbound IPSEC operations.

> > >          *

> > >          *  @see odp_ipsec_in(), odp_ipsec_in_enq()

> > >          */

> > > -       odp_ipsec_op_mode_t op_mode;

> > > +       odp_ipsec_op_mode_t inbound_mode;

> > > +

> > > +       /** Outbound IPSEC operation mode. Application selects which

> mode

> > > +        *  will be used for outbound IPSEC operations.

> > > +        *

> > > +        *  @see odp_ipsec_out(), odp_ipsec_out_enq(),

> > > odp_ipsec_out_inline()

> > > +        */

> > > +       odp_ipsec_op_mode_t outbound_mode;

> > >

> > >         /** Maximum number of IPSEC SAs that application will use

> > >          * simultaneously */

> > > @@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {

> > >         /** IPSEC inbound processing configuration */

> > >         odp_ipsec_inbound_config_t inbound;

> > >

> > > +       /** IPSEC outbound processing configuration */

> > > +       odp_ipsec_outbound_config_t outbound;

> > > +

> > >  } odp_ipsec_config_t;

> > >

> > >  /**

> > > @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

> > >         ODP_IPSEC_LOOKUP_DISABLED = 0,

> > >

> > >         /** Inbound SA lookup is enabled. Used SPI values must be

> unique.

> > > */

> > > -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> > > +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> > > +

> > > +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> > > +         * destination IP address. Used SPI values must be unique. */

> > > +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

> > >

> > >  } odp_ipsec_lookup_mode_t;

> > >

> > >  /**

> > > + * Result event destination

> > > + */

> > > +typedef enum odp_ipsec_dest_mode_t {

> > > +       /** Destination for IPSEC result events is a queue. */

> > > +       ODP_IPSEC_DEST_QUEUE = 0,

> > > +

> > > +       /** Destination for IPSEC result events is the classifier.

> > > +        *  IPSEC capability 'cls_inline' determines if inline

> > > classification

> > > +        *  is supported. */

> > > +       ODP_IPSEC_DEST_CLS

> > > +

> > > +} odp_ipsec_dest_mode_t;

> > > +

> > > +/**

> > >   * IPSEC Security Association (SA) parameters

> > >   */

> > >  typedef struct odp_ipsec_sa_param_t {

> > > @@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {

> > >         /** SPI value */

> > >         uint32_t spi;

> > >

> > > +       /** Additional inbound SA lookup parameters. Values are

> considered

> > > +        *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup

> mode. */

> > > +       struct {

> > > +               /* v4 or v6 */

> > > +               uint8_t ip_version;

> > >

> >

> > Is this an enum? If not perhaps doc this as 4 = IPv4, 6 = IPv6.

> >

> >

> > > +

> > > +               /* IP destination address (NETWORK ENDIAN) */

> > > +               void    *dst_addr;

> > > +

> > > +       } lookup_param;

> > > +

> > >         /** MTU for outbound IP fragmentation offload

> > >          *

> > >          *  This is the maximum length of IP packets that outbound

> IPSEC

> > > @@ -434,13 +629,32 @@ typedef struct odp_ipsec_sa_param_t {

> > >          */

> > >         uint32_t mtu;

> > >

> > > +       /** Select where IPSEC result events are sent

> > > +        *

> > > +        *  Asynchronous and inline modes generate result events.

> Select

> > > where

> > > +        *  those events are sent. Inbound SAs may choose between a

> queue

> > > or

> > > +        *  the classifier. Outbound SAs must define a queue always.

> > > +        *  The default value is ODP_IPSEC_DEST_QUEUE.

> > > +        */

> > > +       odp_ipsec_dest_mode_t dest_mode;

> > > +

> > >         /** Destination queue for IPSEC events

> > >          *

> > > -        *  Operations in asynchronous mode enqueue resulting events

> into

> > > -        *  this queue.

> > > +        *  Operations in asynchronous or inline mode enqueue resulting

> > > events

> > > +        *  into this queue.

> > >          */

> > >         odp_queue_t dest_queue;

> > >

> > > +       /** Classifier destination CoS for IPSEC result events

> > > +        *

> > > +        *  Result events for successfully decapsulated packets are

> sent to

> > > +        *  classification through this CoS. Other result events are

> sent

> > > to

> > > +        *  'dest_queue'. This field is considered only when

> 'dest_mode' is

> > > +        *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any

> > > pktio

> > > +        *  interface default CoS.

> > > +        */

> > > +       odp_cos_t dest_cos;

> > > +

> > >         /** User defined SA context pointer

> > >          *

> > >          *  User defined context pointer associated with the SA.

> > > @@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {

> > >                 uint32_t all_error;

> > >         };

> > >

> > > +       union {

> > > +               /** Status flags */

> > > +               struct {

> > > +                       /** Packet was processed in inline mode */

> > > +                       uint32_t inline_mode      : 1;

> > > +

> > > +               } flag;

> > > +

> > > +               /** All flag bits */

> > > +               uint32_t all_flag;

> > > +       };

> > > +

> > >  } odp_ipsec_op_status_t;

> > >

> > >  /**

> > > @@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {

> > >  } odp_ipsec_op_param_t;

> > >

> > >  /**

> > > + * Outbound inline IPSEC operation parameters

> > > + */

> > > +typedef struct odp_ipsec_inline_op_param_t {

> > > +       /** Packet output interface for inline output operation

> > > +        *

> > > +        *  Outbound inline IPSEC operation uses this packet IO

> interface

> > > to

> > > +        *  output the packet after a successful IPSEC transformation.

> The

> > > pktio

> > > +        *  must have been configured to operate in inline IPSEC mode.

> > > +        */

> > > +       odp_pktio_t pktio;

> > >

> >

> > Shouldn't we have an option for this to be output to an odp_tm_queue_t

> > instead of a pktio?

> > Also, for output are there any controls over which odp_pktout_queue_t is

> > used, or is this undefined?

> >

> >

> > > +

> > > +       /** Outer headers for inline output operation

> > > +        *

> > > +        *  Outbound inline IPSEC operation uses this information to

> > > prepend

> > > +        *  outer headers to the IPSEC packet before sending it out.

> > > +        */

> > > +       struct {

> > > +               /** Points to first byte of outer headers to be copied

> in

> > > +                *  front of the outgoing IPSEC packet. Implementation

> > > copies

> > > +                *  the headers during odp_ipsec_out_inline() call. */

> > > +               uint8_t *ptr;

> > >

> >

> > Should this be an odp_packet_t rather than a raw set of bytes? The

> > rationale for making this an odp_packet_t is that this could then be a

> > packet reference which can be used for (re)transmit tracking. For input

> > having this be raw bytes makes sense since the ODP implementation is

> > controlling their allocation and placement, however for output how is the

> > application going to allocate these? Presumably it would create a header

> > via odp_packet_alloc() and then use odp_packet_data() to get this

> address,

> > but that seems cumbersome compared to simply passing the odp_packet_t

> > directly. The alternative would seem to require that the application use

> > some sort of odp_shm_reserve() call, but that gets awkward and is far

> less

> > efficient than packet allocation, which is a fastpath operation.

> >

> >

> > > +

> > > +               /** Outer header length in bytes */

> > > +               uint32_t len;

> > > +       } outer_hdr;

> > > +

> > > +} odp_ipsec_inline_op_param_t;

> > > +

> > > +/**

> > >   * IPSEC operation result for a packet

> > >   */

> > >  typedef struct odp_ipsec_packet_result_t {

> > > @@ -752,6 +1007,23 @@ typedef struct odp_ipsec_packet_result_t {

> > >          */

> > >         odp_ipsec_sa_t sa;

> > >

> > > +       /** Packet outer header status before inbound inline

> processing.

> > > +        *  This is valid only when status.flag.inline_mode is set.

> > > +        */

> > > +       struct {

> > > +               /** Points to the first byte of retained outer headers.

> > > These

> > > +                *  headers are stored in a contiquous, per packet,

> > > +                *  implementation specific memory space. Since the

> memory

> > > space

> > > +                *  may overlap with e.g. packet head/tailroom, the

> content

> > > +                *  becomes invalid if packet data storage is modified

> in

> > > +                *  anyway. The memory space may not be sharable to

> other

> > > +                *  threads. */

> > > +               uint8_t *ptr;

> > > +

> > > +               /** Outer header length in bytes */

> > > +               uint32_t len;

> > > +       } outer_hdr;

> > > +

> > >  } odp_ipsec_packet_result_t;

> > >

> > >  /**

> > > @@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {

> > >          *  at least 'num_pkt' elements.

> > >          *

> > >          *  Each successfully transformed packet has a valid value for

> > > these

> > > -        *  meta-data:

> > > +        *  meta-data regardless of the inner packet parse

> configuration.

> > >

> >

> > Vestigial typo. It's metadata, not meta-data. Might as well clean up

> here.

> >

> >

> > > +        *  (odp_ipsec_inbound_config_t):

> > >          *  * L3 offset: Offset to the first byte of the (outmost) IP

> > > header

> > > -        *  * L4 offset: Offset to the first byte of the valid and

> known L4

> > > -        *               header (immediately following the IP header).

> > > -        *  * Various flags about L3 and L4 layers:

> > > -        *               has_l3, has_l4, has_ipv4, has_ipv6,

> has_ipfrag,

> > > -        *               has_ipsec, has_udp, has_tcp, etc depending on

> > > -        *               the resulted packet format

> > > +        *  * pktio:     For inbound inline IPSEC processed packets,

> > > original

> > > +        *               packet input interface

> > >          *

> > > -        * @see odp_packet_l3_offset(), odp_packet_l4_offset(),

> > > -        *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),

> > > -        *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()

> > > +        *  Other meta-data for parse results and error checks depend

> on

> > > +        *  configuration (selected parse and error check levels).

> > >          */

> > >         odp_packet_t *pkt;

> > >

> > > @@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t

> *input,

> > >  /**

> > >   * Inbound asynchronous IPSEC operation

> > >   *

> > > - * This operation does inbound IPSEC processing in asynchronous mode

> > > - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise

> identically

> > > to

> > > - * odp_ipsec_in(), but outputs all results through one or more

> > > - * ODP_EVENT_IPSEC_RESULT events with the following ordering

> > > considerations.

> > > + * This operation does inbound IPSEC processing in asynchronous mode.

> It

> > > + * processes packets otherwise identically to odp_ipsec_in(), but

> outputs

> > > all

> > > + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> > > following

> > > + * ordering considerations.

> > >   *

> > >   * Asynchronous mode maintains (operation input) packet order per SA

> when

> > >   * application calls the operation within an ordered or atomic

> scheduler

> > > context

> > > @@ -928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t

> *input,

> > >   * events for the same SA are enqueued in order, and packet handles

> (for

> > > the

> > >   * same SA) are stored in order within an event.

> > >   *

> > > + * The function may be used also in inline processing mode, e.g. for

> IPSEC

> > > + * packets for which inline processing is not possible. Packets for

> the

> > > same SA

> > > + * may be processed simultaneously in both modes (initiated by this

> > > function

> > > + * and inline operation).

> > > + *

> > >   * @param         input   Operation input parameters

> > >   *

> > >   * @return Number of input packets consumed (0 ... input.num_pkt)

> > > @@ -940,10 +1213,10 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> > > *input);

> > >  /**

> > >   * Outbound asynchronous IPSEC operation

> > >   *

> > > - * This operation does outbound IPSEC processing in asynchronous mode

> > > - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise

> identically

> > > to

> > > - * odp_ipsec_out(), but outputs all results through one or more

> > > - * ODP_EVENT_IPSEC_RESULT events with the following ordering

> > > considerations.

> > > + * This operation does outbound IPSEC processing in asynchronous

> mode. It

> > > + * processes packets otherwise identically to odp_ipsec_out(), but

> > > outputs all

> > > + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> > > following

> > > + * ordering considerations.

> > >   *

> > >   * Asynchronous mode maintains (operation input) packet order per SA

> when

> > >   * application calls the operation within an ordered or atomic

> scheduler

> > > context

> > > @@ -953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> > > *input);

> > >   * events for the same SA are enqueued in order, and packet handles

> (for

> > > the

> > >   * same SA) are stored in order within an event.

> > >   *

> > > + * The function may be used also in inline processing mode, e.g. for

> IPSEC

> > > + * packets for which inline processing is not possible.

> > > + *

> > >   * @param         input   Operation input parameters

> > >   *

> > >   * @return Number of input packets consumed (0 ... input.num_pkt)

> > > @@ -963,6 +1239,28 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t

> > > *input);

> > >  int odp_ipsec_out_enq(const odp_ipsec_op_param_t *input);

> > >

> > >  /**

> > > + * Outbound inline IPSEC operation

> > > + *

> > > + * This operation does outbound inline IPSEC processing for the

> packets.

> > > It's

> > > + * otherwise identical to odp_ipsec_out_enq(), but outputs all

> > > successfully

> > > + * transformed packets to the specified output interface, instead of

> > > generating

> > > + * result events for those.

> > > + *

> > > + * Inline operation parameters are defined per packet. The array of

> > > parameters

> > > + * must have 'op_param.num_pkt' elements and is pointed to by

> > > 'inline_param'.

> > > + *

> > > + * @param         op_param      Operation parameters

> > > + * @param         inline_param  Outbound inline operation specific

> > > parameters

> > > + *

> > > + * @return Number of packets consumed (0 ... op_param.num_pkt)

> > > + * @retval <0     On failure

> > > + *

> > > + * @see odp_ipsec_out_enq()

> > > + */

> > > +int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,

> > > +                        const odp_ipsec_inline_op_param_t

> *inline_param);

> > > +

> > > +/**

> > >   * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event

> > >   *

> > >   * Copies IPSEC operation results from an event. The event must be of

> > > diff --git a/include/odp/api/spec/packet_io.h

> > > b/include/odp/api/spec/packet_io.h

> > > index cec1f22..8802089 100644

> > > --- a/include/odp/api/spec/packet_io.h

> > > +++ b/include/odp/api/spec/packet_io.h

> > > @@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {

> > >          * interface capability before enabling the same. */

> > >         odp_bool_t enable_loop;

> > >

> > > +       /** Inbound IPSEC inlined with packet input

> > > +        *

> > > +        *  Enable/disable inline inbound IPSEC operation. When enabled

> > > packet

> > > +        *  input directs all IPSEC packets automatically to IPSEC

> inbound

> > > +        *  processing. IPSEC configuration is done through the IPSEC

> API.

> > > +        *  Packets that are not (recognized as) IPSEC are processed

> > > +        *  according to the packet input configuration.

> > > +        *

> > > +        *  0: Disable inbound IPSEC inline operation (default)

> > > +        *  1: Enable inbound IPSEC inline operation

> > > +        *

> > > +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> > > +        */

> > > +       odp_bool_t inbound_ipsec;

> > > +

> > > +       /** Outbound IPSEC inlined with packet output

> > > +        *

> > > +        *  Enable/disable inline outbound IPSEC operation. When

> enabled

> > > IPSEC

> > > +        *  outbound processing can send outgoing IPSEC packets

> directly

> > > +        *  to the pktio interface for output. IPSEC configuration is

> done

> > > +        *  through the IPSEC API.

> > > +        *

> > > +        *  Outbound IPSEC inline operation cannot be combined with

> traffic

> > > +        *  manager (ODP_PKTOUT_MODE_TM).

> > >

> >

> > We need to fix this since TM is likely to be the preferred TX mode, just

> as

> > the classifier is the preferred RX mode. There's no reason why IPsec

> > traffic shouldn't also be subject to TM policies.

> >

> >

> > > +        *

> > > +        *  0: Disable outbound IPSEC inline operation (default)

> > > +        *  1: Enable outbound IPSEC inline operation

> > > +        *

> > > +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> > > +        */

> > > +       odp_bool_t outbound_ipsec;

> > > +

> > >  } odp_pktio_config_t;

> > >

> > >  /**

> > > --

> > > 2.8.1

> > >

> > >

>

Regards,
Bala

On 21 March 2017 at 19:47, Petri Savolainen <petri.savolainen@linaro.org> wrote:
>

> Added support for inline IPSEC processing on packet input and

> output. Inline mode IPSEC and traffic manager cannot be enabled

> (currently) on the same pktio interface.

>

> Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

> ---

>  include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++++++++---

>  include/odp/api/spec/packet_io.h |  32 ++++

>  2 files changed, 355 insertions(+), 25 deletions(-)

>

> diff --git a/include/odp/api/spec/ipsec.h b/include/odp/api/spec/ipsec.h

> index e951e49..23d02cf 100644

> --- a/include/odp/api/spec/ipsec.h

> +++ b/include/odp/api/spec/ipsec.h

> @@ -19,6 +19,8 @@ extern "C" {

>  #endif

>

>  #include <odp/api/crypto.h>

> +#include <odp/api/packet_io.h>

> +#include <odp/api/classification.h>

>

>  /** @defgroup odp_ipsec ODP IPSEC

>   *  Operations of IPSEC API.

> @@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {

>           * Application uses asynchronous IPSEC operations,

>           * which return results via events.

>           */

> -       ODP_IPSEC_OP_MODE_ASYNC

> +       ODP_IPSEC_OP_MODE_ASYNC,

> +

> +       /** Inline IPSEC operation

> +         *

> +         * Packet input/output is connected directly to IPSEC inbound/outbound

> +         * processing. Application uses asynchronous or inline IPSEC

> +         * operations.

> +         */

> +       ODP_IPSEC_OP_MODE_INLINE,

> +

> +       /** IPSEC is disabled in inbound / outbound direction */

> +       ODP_IPSEC_OP_MODE_DISABLED

>

>  } odp_ipsec_op_mode_t;

>

>  /**

> + * Protocol layers in IPSEC configuration

> + */

> +typedef enum odp_ipsec_proto_layer_t {

> +       /** No layers */

> +       ODP_IPSEC_LAYER_NONE = 0,

> +

> +       /** Layer L2 protocols (Ethernet, VLAN, etc) */

> +       ODP_IPSEC_LAYER_L2,

> +

> +       /** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

> +       ODP_IPSEC_LAYER_L3,

> +

> +       /** Layer L4 protocols (UDP, TCP, SCTP) */

> +       ODP_IPSEC_LAYER_L4,

> +

> +       /** All layers */

> +       ODP_IPSEC_LAYER_ALL

> +

> +} odp_ipsec_proto_layer_t;

> +

> +/**

>   * Configuration options for IPSEC inbound processing

>   */

>  typedef struct odp_ipsec_inbound_config_t {

> @@ -77,9 +111,113 @@ typedef struct odp_ipsec_inbound_config_t {

>                 uint32_t max;

>         } spi;

>

> +       /** Retain outer headers

> +        *

> +        *  Select up to which protocol layer (at least) outer headers are

> +        *  retained in inbound inline processing. Default value is

> +        *  ODP_IPSEC_LAYER_NONE.

> +        *

> +        *  ODP_IPSEC_LAYER_NONE: Application does not require any outer

> +        *                        headers to be retained.

> +        *

> +        *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.

> +        *

> +        *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3, otherwise the

> +        *                        same as ODP_IPSEC_LAYER_ALL.

> +        *

> +        *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4, otherwise the

> +        *                        same as ODP_IPSEC_LAYER_ALL.

> +        *

> +        *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before IPSEC are

> +        *                        retained. In transport mode, all headers

> +        *                        before IP (carrying IPSEC) are retained.

> +        *

> +        */

> +       odp_ipsec_proto_layer_t retain_outer;

> +

> +       /** Parse packet headers in IPSEC payload

> +        *

> +        *  Select header parsing level after inbound processing. Packet headers

> +        *  in IPSEC payload must be parsed (at least) up to this level.

> +        *  Default value is ODP_IPSEC_LAYER_NONE.

> +        *

> +        *  Note that IPSec payload is never a L2 packet (ODP_IPSEC_LAYER_L2

> +        *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC payload

> +        *  starts after IP header (ODP_IPSEC_LAYER_L3 equals

> +        *  ODP_IPSEC_LAYER_NONE).

> +        */

> +       odp_ipsec_proto_layer_t parse;

> +

> +       /** Flags to control IPSEC payload data checks up to the selected parse

> +        *  level. */

> +       union {

> +               struct {

> +                       /** Check IPv4 header checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t ipv4_chksum   : 1;

> +

> +                       /** Check UDP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t udp_chksum    : 1;

> +

> +                       /** Check TCP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t tcp_chksum    : 1;

> +

> +                       /** Check SCTP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t sctp_chksum   : 1;

> +               } check;

> +

> +               /** All bits of the bit field structure

> +                 *

> +                 * This field can be used to set/clear all flags, or bitwise

> +                 * operations over the entire structure. */

> +               uint32_t all_check;

> +       };

> +

>  } odp_ipsec_inbound_config_t;

>

>  /**

> + * Configuration options for IPSEC outbound processing

> + */

> +typedef struct odp_ipsec_outbound_config_t {

> +       /** Flags to control L3/L4 checksum insertion as part of outbound

> +        *  packet processing. Packet must have set with valid L3/L4 offsets.

> +        *  Checksum configuration is ignored for packets that checksum cannot

> +        *  be computed for (e.g. IPv4 fragments). Application may use a packet

> +        *  metadata flag to disable checksum insertion per packet bases.

> +        */

> +       union {

> +               struct {

> +                       /** Insert IPv4 header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_ipv4   : 1;

> +

> +                       /** Insert UDP header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_udp    : 1;

> +

> +                       /** Insert TCP header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_tcp    : 1;

> +

> +                       /** Insert SCTP header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_sctp   : 1;

> +

> +               } chksum;

> +

> +               /** All bits of the bit field structure

> +                 *

> +                 * This field can be used to set/clear all flags, or bitwise

> +                 * operations over the entire structure. */

> +               uint32_t all_chksum;

> +       };

> +

> +} odp_ipsec_outbound_config_t;

> +

> +/**

>   * IPSEC capability

>   */

>  typedef struct odp_ipsec_capability_t {

> @@ -102,6 +240,24 @@ typedef struct odp_ipsec_capability_t {

>          */

>         uint8_t op_mode_async;

>

> +       /** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE) support

> +        *

> +        *  0: Inline IPSEC operation is not supported

> +        *  1: Inline IPSEC operation is supported

> +        *  2: Inline IPSEC operation is supported and preferred

> +        */

> +       uint8_t op_mode_inline;

> +

> +       /** Support of inline classification (ODP_IPSEC_DEST_CLS) for resulting

> +        *  inbound packets.

> +        *

> +        *  0: Inline classification of resulting packets is not supported

> +        *  1: Inline classification of resulting packets is supported

> +        *  2: Inline classification of resulting packets is supported and

> +        *     preferred

> +        */

> +       uint8_t cls_inline;

> +

>         /** Soft expiry limit in seconds support

>          *

>          *  0: Limit is not supported

> @@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {

>   * IPSEC configuration options

>   */

>  typedef struct odp_ipsec_config_t {

> -       /** IPSEC operation mode. Application selects which mode (sync or async)

> -        *  will be used for IPSEC operations.

> +       /** Inbound IPSEC operation mode. Application selects which mode

> +        *  will be used for inbound IPSEC operations.

>          *

>          *  @see odp_ipsec_in(), odp_ipsec_in_enq()

>          */

> -       odp_ipsec_op_mode_t op_mode;

> +       odp_ipsec_op_mode_t inbound_mode;

> +

> +       /** Outbound IPSEC operation mode. Application selects which mode

> +        *  will be used for outbound IPSEC operations.

> +        *

> +        *  @see odp_ipsec_out(), odp_ipsec_out_enq(), odp_ipsec_out_inline()

> +        */

> +       odp_ipsec_op_mode_t outbound_mode;

>

>         /** Maximum number of IPSEC SAs that application will use

>          * simultaneously */

> @@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {

>         /** IPSEC inbound processing configuration */

>         odp_ipsec_inbound_config_t inbound;

>

> +       /** IPSEC outbound processing configuration */

> +       odp_ipsec_outbound_config_t outbound;

> +

>  } odp_ipsec_config_t;

>

>  /**

> @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

>         ODP_IPSEC_LOOKUP_DISABLED = 0,

>

>         /** Inbound SA lookup is enabled. Used SPI values must be unique. */

> -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> +

> +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> +         * destination IP address. Used SPI values must be unique. */

> +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

>

>  } odp_ipsec_lookup_mode_t;


odp_ipsec_lookup_mode_t is not added either in odp_ipsec_config() or
in odp_ipsec_capability().
I believe this should be added in both these struct?

>

>

>  /**

> + * Result event destination

> + */

> +typedef enum odp_ipsec_dest_mode_t {

> +       /** Destination for IPSEC result events is a queue. */

> +       ODP_IPSEC_DEST_QUEUE = 0,

> +

> +       /** Destination for IPSEC result events is the classifier.

> +        *  IPSEC capability 'cls_inline' determines if inline classification

> +        *  is supported. */

> +       ODP_IPSEC_DEST_CLS

> +

> +} odp_ipsec_dest_mode_t;


Should'nt we add a dest_mode ODP_IPSEC_DEST_PKTIO for outbound inline
when the packet is sent out through interface directly.

>

> +

> +/**

>   * IPSEC Security Association (SA) parameters

>   */

>  typedef struct odp_ipsec_sa_param_t {

> @@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {

>         /** SPI value */

>         uint32_t spi;

>

> +       /** Additional inbound SA lookup parameters. Values are considered

> +        *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */

> +       struct {

> +               /* v4 or v6 */

> +               uint8_t ip_version;

> +

> +               /* IP destination address (NETWORK ENDIAN) */

> +               void    *dst_addr;

> +

> +       } lookup_param;

> +

>         /** MTU for outbound IP fragmentation offload

>          *

>          *  This is the maximum length of IP packets that outbound IPSEC

> @@ -434,13 +629,32 @@ typedef struct odp_ipsec_sa_param_t {

>          */

>         uint32_t mtu;

>

> +       /** Select where IPSEC result events are sent

> +        *

> +        *  Asynchronous and inline modes generate result events. Select where

> +        *  those events are sent. Inbound SAs may choose between a queue or

> +        *  the classifier. Outbound SAs must define a queue always.

> +        *  The default value is ODP_IPSEC_DEST_QUEUE.

> +        */

> +       odp_ipsec_dest_mode_t dest_mode;

> +

>         /** Destination queue for IPSEC events

>          *

> -        *  Operations in asynchronous mode enqueue resulting events into

> l-        *  this queue.

> +        *  Operations in asynchronous or inline mode enqueue resulting events

> +        *  into this queue.

>          */

>         odp_queue_t dest_queue;

>

> +       /** Classifier destination CoS for IPSEC result events

> +        *

> +        *  Result events for successfully decapsulated packets are sent to

> +        *  classification through this CoS. Other result events are sent to

> +        *  'dest_queue'. This field is considered only when 'dest_mode' is

> +        *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any pktio

> +        *  interface default CoS.

> +        */

> +       odp_cos_t dest_cos;

> +

>         /** User defined SA context pointer

>          *

>          *  User defined context pointer associated with the SA.

> @@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {

>                 uint32_t all_error;

>         };

>

> +       union {

> +               /** Status flags */

> +               struct {

> +                       /** Packet was processed in inline mode */

> +                       uint32_t inline_mode      : 1;

> +

> +               } flag;

> +

> +               /** All flag bits */

> +               uint32_t all_flag;

> +       };

> +

>  } odp_ipsec_op_status_t;

>

>  /**

> @@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {

>  } odp_ipsec_op_param_t;

>

>  /**

> + * Outbound inline IPSEC operation parameters

> + */

> +typedef struct odp_ipsec_inline_op_param_t {

> +       /** Packet output interface for inline output operation

> +        *

> +        *  Outbound inline IPSEC operation uses this packet IO interface to

> +        *  output the packet after a successful IPSEC transformation. The pktio

> +        *  must have been configured to operate in inline IPSEC mode.

> +        */

> +       odp_pktio_t pktio;

> +

> +       /** Outer headers for inline output operation

> +        *

> +        *  Outbound inline IPSEC operation uses this information to prepend

> +        *  outer headers to the IPSEC packet before sending it out.

> +        */

> +       struct {

> +               /** Points to first byte of outer headers to be copied in

> +                *  front of the outgoing IPSEC packet. Implementation copies

> +                *  the headers during odp_ipsec_out_inline() call. */

> +               uint8_t *ptr;

> +

> +               /** Outer header length in bytes */

> +               uint32_t len;

> +       } outer_hdr;

> +

> +} odp_ipsec_inline_op_param_t;

> +

> +/**

>   * IPSEC operation result for a packet

>   */

>  typedef struct odp_ipsec_packet_result_t {

> @@ -752,6 +1007,23 @@ typedef struct odp_ipsec_packet_result_t {

>          */

>         odp_ipsec_sa_t sa;

>

> +       /** Packet outer header status before inbound inline processing.

> +        *  This is valid only when status.flag.inline_mode is set.

> +        */

> +       struct {

> +               /** Points to the first byte of retained outer headers. These

> +                *  headers are stored in a contiquous, per packet,

> +                *  implementation specific memory space. Since the memory space

> +                *  may overlap with e.g. packet head/tailroom, the content

> +                *  becomes invalid if packet data storage is modified in

> +                *  anyway. The memory space may not be sharable to other

> +                *  threads. */

> +               uint8_t *ptr;

> +

> +               /** Outer header length in bytes */

> +               uint32_t len;

> +       } outer_hdr;

> +

>  } odp_ipsec_packet_result_t;

>

>  /**

> @@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {

>          *  at least 'num_pkt' elements.

>          *

>          *  Each successfully transformed packet has a valid value for these

> -        *  meta-data:

> +        *  meta-data regardless of the inner packet parse configuration.

> +        *  (odp_ipsec_inbound_config_t):

>          *  * L3 offset: Offset to the first byte of the (outmost) IP header

> -        *  * L4 offset: Offset to the first byte of the valid and known L4

> -        *               header (immediately following the IP header).

> -        *  * Various flags about L3 and L4 layers:

> -        *               has_l3, has_l4, has_ipv4, has_ipv6, has_ipfrag,

> -        *               has_ipsec, has_udp, has_tcp, etc depending on

> -        *               the resulted packet format

> +        *  * pktio:     For inbound inline IPSEC processed packets, original

> +        *               packet input interface

>          *

> -        * @see odp_packet_l3_offset(), odp_packet_l4_offset(),

> -        *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),

> -        *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()

> +        *  Other meta-data for parse results and error checks depend on

> +        *  configuration (selected parse and error check levels).

>          */

>         odp_packet_t *pkt;

>

> @@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>  /**

>   * Inbound asynchronous IPSEC operation

>   *

> - * This operation does inbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to

> - * odp_ipsec_in(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.

> + * This operation does inbound IPSEC processing in asynchronous mode. It

> + * processes packets otherwise identically to odp_ipsec_in(), but outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler context

> @@ -928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>   * events for the same SA are enqueued in order, and packet handles (for the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for IPSEC

> + * packets for which inline processing is not possible. Packets for the same SA

> + * may be processed simultaneously in both modes (initiated by this function

> + * and inline operation).

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt)

> @@ -940,10 +1213,10 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>  /**

>   * Outbound asynchronous IPSEC operation

>   *

> - * This operation does outbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to

> - * odp_ipsec_out(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.

> + * This operation does outbound IPSEC processing in asynchronous mode. It

> + * processes packets otherwise identically to odp_ipsec_out(), but outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler context

> @@ -953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>   * events for the same SA are enqueued in order, and packet handles (for the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for IPSEC

> + * packets for which inline processing is not possible.

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt)

> @@ -963,6 +1239,28 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>  int odp_ipsec_out_enq(const odp_ipsec_op_param_t *input);

>

>  /**

> + * Outbound inline IPSEC operation

> + *

> + * This operation does outbound inline IPSEC processing for the packets. It's

> + * otherwise identical to odp_ipsec_out_enq(), but outputs all successfully

> + * transformed packets to the specified output interface, instead of generating

> + * result events for those.

> + *

> + * Inline operation parameters are defined per packet. The array of parameters

> + * must have 'op_param.num_pkt' elements and is pointed to by 'inline_param'.

> + *

> + * @param         op_param      Operation parameters

> + * @param         inline_param  Outbound inline operation specific parameters

> + *

> + * @return Number of packets consumed (0 ... op_param.num_pkt)

> + * @retval <0     On failure

> + *

> + * @see odp_ipsec_out_enq()

> + */

> +int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,

> +                        const odp_ipsec_inline_op_param_t *inline_param);

> +

> +/**

>   * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event

>   *

>   * Copies IPSEC operation results from an event. The event must be of

> diff --git a/include/odp/api/spec/packet_io.h b/include/odp/api/spec/packet_io.h

> index cec1f22..8802089 100644

> --- a/include/odp/api/spec/packet_io.h

> +++ b/include/odp/api/spec/packet_io.h

> @@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {

>          * interface capability before enabling the same. */

>         odp_bool_t enable_loop;

>

> +       /** Inbound IPSEC inlined with packet input

> +        *

> +        *  Enable/disable inline inbound IPSEC operation. When enabled packet

> +        *  input directs all IPSEC packets automatically to IPSEC inbound

> +        *  processing. IPSEC configuration is done through the IPSEC API.

> +        *  Packets that are not (recognized as) IPSEC are processed

> +        *  according to the packet input configuration.

> +        *

> +        *  0: Disable inbound IPSEC inline operation (default)

> +        *  1: Enable inbound IPSEC inline operation

> +        *

> +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +        */

> +       odp_bool_t inbound_ipsec;

> +

> +       /** Outbound IPSEC inlined with packet output

> +        *

> +        *  Enable/disable inline outbound IPSEC operation. When enabled IPSEC

> +        *  outbound processing can send outgoing IPSEC packets directly

> +        *  to the pktio interface for output. IPSEC configuration is done

> +        *  through the IPSEC API.

> +        *

> +        *  Outbound IPSEC inline operation cannot be combined with traffic

> +        *  manager (ODP_PKTOUT_MODE_TM).

> +        *

> +        *  0: Disable outbound IPSEC inline operation (default)

> +        *  1: Enable outbound IPSEC inline operation

> +        *

> +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +        */

> +       odp_bool_t outbound_ipsec;

> +

>  } odp_pktio_config_t;

>

>  /**

> --

> 2.8.1

>

Few initial comments:

- For Mixing inline and async processing must maintain same SA context in HW and implementation. In that case, we should shift to async SA creation so that while creating an SA, HW    interface can return the context to sw and Async processing can be done for that context.
- Why we want to add different level of header retention, we can add retain header or discard.
- Why parsing is optional and layer wise. Parsing should be same as on the pktio interface. Even in the lookaside APIs L3 parsing was mandatory. Otherwise application will have to do the parsing on its own. Moreover, application will never know L3 protocol if there are no parse results.
- In case outer headers are retained in the packet, hardware will parse packet based outer headers not inner. Same goes for classification.
- Checksum configuration (For inbound as well as outbound) can be per flow basis, as requirement can be different for different flows.
- Support for ESP/AH and Tunnel/Transport mode should be exposed via capability.
- Inline SA lookup should also support 3 tuple lookups (SIP, DIP, SPI).
- Why there are separate lookup prams in SA, when SA tunnel params already include SIP and DIP?
- What about enabling HASH distribution post IPSEC for FAT tunnel use case.
- In per SA params, if dest_mode is ODP_IPSEC_DEST_CLS, why output is to a new CoS, why not to default CoS of that pktio. Are we saying that we will have two level of classification on that oktio, one before IPsec and one after that?
- Who holds the responsibility of freeing the l2hdr in odp_ipsec_inline_op_param_t post transmission? IUs it mandated to be inside the packet? The why not just use l2_offset?

Regards
Nikhil

-----Original Message-----
From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Petri Savolainen

Sent: Tuesday, March 21, 2017 7:47 PM
To: lng-odp@lists.linaro.org
Subject: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

Added support for inline IPSEC processing on packet input and output. Inline mode IPSEC and traffic manager cannot be enabled
(currently) on the same pktio interface.

Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

---
 include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++++++++---
 include/odp/api/spec/packet_io.h |  32 ++++
 2 files changed, 355 insertions(+), 25 deletions(-)

--
2.8.1diff --git a/include/odp/api/spec/ipsec.h b/include/odp/api/spec/ipsec.h index e951e49..23d02cf 100644
--- a/include/odp/api/spec/ipsec.h
+++ b/include/odp/api/spec/ipsec.h
@@ -19,6 +19,8 @@ extern "C" {
 #endif
 
 #include <odp/api/crypto.h>
+#include <odp/api/packet_io.h>
+#include <odp/api/classification.h>
 
 /** @defgroup odp_ipsec ODP IPSEC
  *  Operations of IPSEC API.
@@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {
 	  * Application uses asynchronous IPSEC operations,
 	  * which return results via events.
 	  */
-	ODP_IPSEC_OP_MODE_ASYNC
+	ODP_IPSEC_OP_MODE_ASYNC,
+
+	/** Inline IPSEC operation
+	  *
+	  * Packet input/output is connected directly to IPSEC inbound/outbound
+	  * processing. Application uses asynchronous or inline IPSEC
+	  * operations.
+	  */
+	ODP_IPSEC_OP_MODE_INLINE,
+
+	/** IPSEC is disabled in inbound / outbound direction */
+	ODP_IPSEC_OP_MODE_DISABLED
 
 } odp_ipsec_op_mode_t;
 
 /**
+ * Protocol layers in IPSEC configuration  */ typedef enum 
+odp_ipsec_proto_layer_t {
+	/** No layers */
+	ODP_IPSEC_LAYER_NONE = 0,
+
+	/** Layer L2 protocols (Ethernet, VLAN, etc) */
+	ODP_IPSEC_LAYER_L2,
+
+	/** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */
+	ODP_IPSEC_LAYER_L3,
+
+	/** Layer L4 protocols (UDP, TCP, SCTP) */
+	ODP_IPSEC_LAYER_L4,
+
+	/** All layers */
+	ODP_IPSEC_LAYER_ALL
+
+} odp_ipsec_proto_layer_t;
+
+/**
  * Configuration options for IPSEC inbound processing
  */
 typedef struct odp_ipsec_inbound_config_t { @@ -77,9 +111,113 @@ typedef struct odp_ipsec_inbound_config_t {
 		uint32_t max;
 	} spi;
 
+	/** Retain outer headers
+	 *
+	 *  Select up to which protocol layer (at least) outer headers are
+	 *  retained in inbound inline processing. Default value is
+	 *  ODP_IPSEC_LAYER_NONE.
+	 *
+	 *  ODP_IPSEC_LAYER_NONE: Application does not require any outer
+	 *                        headers to be retained.
+	 *
+	 *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.
+	 *
+	 *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3, otherwise the
+	 *                        same as ODP_IPSEC_LAYER_ALL.
+	 *
+	 *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4, otherwise the
+	 *                        same as ODP_IPSEC_LAYER_ALL.
+	 *
+	 *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before IPSEC are
+	 *                        retained. In transport mode, all headers
+	 *                        before IP (carrying IPSEC) are retained.
+	 *
+	 */
+	odp_ipsec_proto_layer_t retain_outer;
+
+	/** Parse packet headers in IPSEC payload
+	 *
+	 *  Select header parsing level after inbound processing. Packet headers
+	 *  in IPSEC payload must be parsed (at least) up to this level.
+	 *  Default value is ODP_IPSEC_LAYER_NONE.
+	 *
+	 *  Note that IPSec payload is never a L2 packet (ODP_IPSEC_LAYER_L2
+	 *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC payload
+	 *  starts after IP header (ODP_IPSEC_LAYER_L3 equals
+	 *  ODP_IPSEC_LAYER_NONE).
+	 */
+	odp_ipsec_proto_layer_t parse;
+
+	/** Flags to control IPSEC payload data checks up to the selected parse
+	 *  level. */
+	union {
+		struct {
+			/** Check IPv4 header checksum in IPSEC payload.
+			 *  Default value is 0. */
+			uint32_t ipv4_chksum   : 1;
+
+			/** Check UDP checksum in IPSEC payload.
+			 *  Default value is 0. */
+			uint32_t udp_chksum    : 1;
+
+			/** Check TCP checksum in IPSEC payload.
+			 *  Default value is 0. */
+			uint32_t tcp_chksum    : 1;
+
+			/** Check SCTP checksum in IPSEC payload.
+			 *  Default value is 0. */
+			uint32_t sctp_chksum   : 1;
+		} check;
+
+		/** All bits of the bit field structure
+		  *
+		  * This field can be used to set/clear all flags, or bitwise
+		  * operations over the entire structure. */
+		uint32_t all_check;
+	};
+
 } odp_ipsec_inbound_config_t;
 
 /**
+ * Configuration options for IPSEC outbound processing  */ typedef 
+struct odp_ipsec_outbound_config_t {
+	/** Flags to control L3/L4 checksum insertion as part of outbound
+	 *  packet processing. Packet must have set with valid L3/L4 offsets.
+	 *  Checksum configuration is ignored for packets that checksum cannot
+	 *  be computed for (e.g. IPv4 fragments). Application may use a packet
+	 *  metadata flag to disable checksum insertion per packet bases.
+	 */
+	union {
+		struct {
+			/** Insert IPv4 header checksum on the payload packet
+			 *  before IPSEC transformation. Default value is 0. */
+			uint32_t inner_ipv4   : 1;
+
+			/** Insert UDP header checksum on the payload packet
+			 *  before IPSEC transformation. Default value is 0. */
+			uint32_t inner_udp    : 1;
+
+			/** Insert TCP header checksum on the payload packet
+			 *  before IPSEC transformation. Default value is 0. */
+			uint32_t inner_tcp    : 1;
+
+			/** Insert SCTP header checksum on the payload packet
+			 *  before IPSEC transformation. Default value is 0. */
+			uint32_t inner_sctp   : 1;
+
+		} chksum;
+
+		/** All bits of the bit field structure
+		  *
+		  * This field can be used to set/clear all flags, or bitwise
+		  * operations over the entire structure. */
+		uint32_t all_chksum;
+	};
+
+} odp_ipsec_outbound_config_t;
+
+/**
  * IPSEC capability
  */
 typedef struct odp_ipsec_capability_t { @@ -102,6 +240,24 @@ typedef struct odp_ipsec_capability_t {
 	 */
 	uint8_t op_mode_async;
 
+	/** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE) support
+	 *
+	 *  0: Inline IPSEC operation is not supported
+	 *  1: Inline IPSEC operation is supported
+	 *  2: Inline IPSEC operation is supported and preferred
+	 */
+	uint8_t op_mode_inline;
+
+	/** Support of inline classification (ODP_IPSEC_DEST_CLS) for resulting
+	 *  inbound packets.
+	 *
+	 *  0: Inline classification of resulting packets is not supported
+	 *  1: Inline classification of resulting packets is supported
+	 *  2: Inline classification of resulting packets is supported and
+	 *     preferred
+	 */
+	uint8_t cls_inline;
+
 	/** Soft expiry limit in seconds support
 	 *
 	 *  0: Limit is not supported
@@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {
  * IPSEC configuration options
  */
 typedef struct odp_ipsec_config_t {
-	/** IPSEC operation mode. Application selects which mode (sync or async)
-	 *  will be used for IPSEC operations.
+	/** Inbound IPSEC operation mode. Application selects which mode
+	 *  will be used for inbound IPSEC operations.
 	 *
 	 *  @see odp_ipsec_in(), odp_ipsec_in_enq()
 	 */
-	odp_ipsec_op_mode_t op_mode;
+	odp_ipsec_op_mode_t inbound_mode;
+
+	/** Outbound IPSEC operation mode. Application selects which mode
+	 *  will be used for outbound IPSEC operations.
+	 *
+	 *  @see odp_ipsec_out(), odp_ipsec_out_enq(), odp_ipsec_out_inline()
+	 */
+	odp_ipsec_op_mode_t outbound_mode;
 
 	/** Maximum number of IPSEC SAs that application will use
 	 * simultaneously */
@@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {
 	/** IPSEC inbound processing configuration */
 	odp_ipsec_inbound_config_t inbound;
 
+	/** IPSEC outbound processing configuration */
+	odp_ipsec_outbound_config_t outbound;
+
 } odp_ipsec_config_t;
 
 /**
@@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {
 	ODP_IPSEC_LOOKUP_DISABLED = 0,
 
 	/** Inbound SA lookup is enabled. Used SPI values must be unique. */
-	ODP_IPSEC_LOOKUP_IN_UNIQUE_SA
+	ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,
+
+	/** Inbound SA lookup is enabled. Lookup matches both SPI and
+	  * destination IP address. Used SPI values must be unique. */
+	ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI
 
 } odp_ipsec_lookup_mode_t;
 
 /**
+ * Result event destination
+ */
+typedef enum odp_ipsec_dest_mode_t {
+	/** Destination for IPSEC result events is a queue. */
+	ODP_IPSEC_DEST_QUEUE = 0,
+
+	/** Destination for IPSEC result events is the classifier.
+	 *  IPSEC capability 'cls_inline' determines if inline classification
+	 *  is supported. */
+	ODP_IPSEC_DEST_CLS
+
+} odp_ipsec_dest_mode_t;
+
+/**
  * IPSEC Security Association (SA) parameters
  */
 typedef struct odp_ipsec_sa_param_t {
@@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {
 	/** SPI value */
 	uint32_t spi;
 
+	/** Additional inbound SA lookup parameters. Values are considered
+	 *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */
+	struct {
+		/* v4 or v6 */
+		uint8_t ip_version;
+
+		/* IP destination address (NETWORK ENDIAN) */
+		void    *dst_addr;
+
+	} lookup_param;
+
 	/** MTU for outbound IP fragmentation offload
 	 *
 	 *  This is the maximum length of IP packets that outbound IPSEC @@ -434,13 +629,32 @@ typedef struct odp_ipsec_sa_param_t {
 	 */
 	uint32_t mtu;
 
+	/** Select where IPSEC result events are sent
+	 *
+	 *  Asynchronous and inline modes generate result events. Select where
+	 *  those events are sent. Inbound SAs may choose between a queue or
+	 *  the classifier. Outbound SAs must define a queue always.
+	 *  The default value is ODP_IPSEC_DEST_QUEUE.
+	 */
+	odp_ipsec_dest_mode_t dest_mode;
+
 	/** Destination queue for IPSEC events
 	 *
-	 *  Operations in asynchronous mode enqueue resulting events into
-	 *  this queue.
+	 *  Operations in asynchronous or inline mode enqueue resulting events
+	 *  into this queue.
 	 */
 	odp_queue_t dest_queue;
 
+	/** Classifier destination CoS for IPSEC result events
+	 *
+	 *  Result events for successfully decapsulated packets are sent to
+	 *  classification through this CoS. Other result events are sent to
+	 *  'dest_queue'. This field is considered only when 'dest_mode' is
+	 *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any pktio
+	 *  interface default CoS.
+	 */
+	odp_cos_t dest_cos;
+
 	/** User defined SA context pointer
 	 *
 	 *  User defined context pointer associated with the SA.
@@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {
 		uint32_t all_error;
 	};
 
+	union {
+		/** Status flags */
+		struct {
+			/** Packet was processed in inline mode */
+			uint32_t inline_mode      : 1;
+
+		} flag;
+
+		/** All flag bits */
+		uint32_t all_flag;
+	};
+
 } odp_ipsec_op_status_t;
 
 /**
@@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {  } odp_ipsec_op_param_t;
 
 /**
+ * Outbound inline IPSEC operation parameters  */ typedef struct 
+odp_ipsec_inline_op_param_t {
+	/** Packet output interface for inline output operation
+	 *
+	 *  Outbound inline IPSEC operation uses this packet IO interface to
+	 *  output the packet after a successful IPSEC transformation. The pktio
+	 *  must have been configured to operate in inline IPSEC mode.
+	 */
+	odp_pktio_t pktio;
+
+	/** Outer headers for inline output operation
+	 *
+	 *  Outbound inline IPSEC operation uses this information to prepend
+	 *  outer headers to the IPSEC packet before sending it out.
+	 */
+	struct {
+		/** Points to first byte of outer headers to be copied in
+		 *  front of the outgoing IPSEC packet. Implementation copies
+		 *  the headers during odp_ipsec_out_inline() call. */
+		uint8_t *ptr;
+
+		/** Outer header length in bytes */
+		uint32_t len;
+	} outer_hdr;
+
+} odp_ipsec_inline_op_param_t;
+
+/**
  * IPSEC operation result for a packet
  */
 typedef struct odp_ipsec_packet_result_t { @@ -752,6 +1007,23 @@ typedef struct odp_ipsec_packet_result_t {
 	 */
 	odp_ipsec_sa_t sa;
 
+	/** Packet outer header status before inbound inline processing.
+	 *  This is valid only when status.flag.inline_mode is set.
+	 */
+	struct {
+		/** Points to the first byte of retained outer headers. These
+		 *  headers are stored in a contiquous, per packet,
+		 *  implementation specific memory space. Since the memory space
+		 *  may overlap with e.g. packet head/tailroom, the content
+		 *  becomes invalid if packet data storage is modified in
+		 *  anyway. The memory space may not be sharable to other
+		 *  threads. */
+		uint8_t *ptr;
+
+		/** Outer header length in bytes */
+		uint32_t len;
+	} outer_hdr;
+
 } odp_ipsec_packet_result_t;
 
 /**
@@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {
 	 *  at least 'num_pkt' elements.
 	 *
 	 *  Each successfully transformed packet has a valid value for these
-	 *  meta-data:
+	 *  meta-data regardless of the inner packet parse configuration.
+	 *  (odp_ipsec_inbound_config_t):
 	 *  * L3 offset: Offset to the first byte of the (outmost) IP header
-	 *  * L4 offset: Offset to the first byte of the valid and known L4
-	 *               header (immediately following the IP header).
-	 *  * Various flags about L3 and L4 layers:
-	 *               has_l3, has_l4, has_ipv4, has_ipv6, has_ipfrag,
-	 *               has_ipsec, has_udp, has_tcp, etc depending on
-	 *               the resulted packet format
+	 *  * pktio:     For inbound inline IPSEC processed packets, original
+	 *               packet input interface
 	 *
-	 * @see odp_packet_l3_offset(), odp_packet_l4_offset(),
-	 *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),
-	 *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()
+	 *  Other meta-data for parse results and error checks depend on
+	 *  configuration (selected parse and error check levels).
 	 */
 	odp_packet_t *pkt;
 
@@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,
 /**
  * Inbound asynchronous IPSEC operation
  *
- * This operation does inbound IPSEC processing in asynchronous mode
- * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to
- * odp_ipsec_in(), but outputs all results through one or more
- * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.
+ * This operation does inbound IPSEC processing in asynchronous mode. 
+ It
+ * processes packets otherwise identically to odp_ipsec_in(), but 
+ outputs all
+ * results through one or more ODP_EVENT_IPSEC_RESULT events with the 
+ following
+ * ordering considerations.
  *
  * Asynchronous mode maintains (operation input) packet order per SA when
  * application calls the operation within an ordered or atomic scheduler context @@ -928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,
  * events for the same SA are enqueued in order, and packet handles (for the
  * same SA) are stored in order within an event.
  *
+ * The function may be used also in inline processing mode, e.g. for 
+ IPSEC
+ * packets for which inline processing is not possible. Packets for the 
+ same SA
+ * may be processed simultaneously in both modes (initiated by this 
+ function
+ * and inline operation).
+ *
  * @param         input   Operation input parameters
  *
  * @return Number of input packets consumed (0 ... input.num_pkt) @@ -940,10 +1213,10 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);
 /**
  * Outbound asynchronous IPSEC operation
  *
- * This operation does outbound IPSEC processing in asynchronous mode
- * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to
- * odp_ipsec_out(), but outputs all results through one or more
- * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.
+ * This operation does outbound IPSEC processing in asynchronous mode. 
+ It
+ * processes packets otherwise identically to odp_ipsec_out(), but 
+ outputs all
+ * results through one or more ODP_EVENT_IPSEC_RESULT events with the 
+ following
+ * ordering considerations.
  *
  * Asynchronous mode maintains (operation input) packet order per SA when
  * application calls the operation within an ordered or atomic scheduler context @@ -953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);
  * events for the same SA are enqueued in order, and packet handles (for the
  * same SA) are stored in order within an event.
  *
+ * The function may be used also in inline processing mode, e.g. for 
+ IPSEC
+ * packets for which inline processing is not possible.
+ *
  * @param         input   Operation input parameters
  *
  * @return Number of input packets consumed (0 ... input.num_pkt) @@ -963,6 +1239,28 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);  int odp_ipsec_out_enq(const odp_ipsec_op_param_t *input);
 
 /**
+ * Outbound inline IPSEC operation
+ *
+ * This operation does outbound inline IPSEC processing for the 
+packets. It's
+ * otherwise identical to odp_ipsec_out_enq(), but outputs all 
+successfully
+ * transformed packets to the specified output interface, instead of 
+generating
+ * result events for those.
+ *
+ * Inline operation parameters are defined per packet. The array of 
+parameters
+ * must have 'op_param.num_pkt' elements and is pointed to by 'inline_param'.
+ *
+ * @param         op_param      Operation parameters
+ * @param         inline_param  Outbound inline operation specific parameters
+ *
+ * @return Number of packets consumed (0 ... op_param.num_pkt)
+ * @retval <0     On failure
+ *
+ * @see odp_ipsec_out_enq()
+ */
+int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,
+			 const odp_ipsec_inline_op_param_t *inline_param);
+
+/**
  * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event
  *
  * Copies IPSEC operation results from an event. The event must be of diff --git a/include/odp/api/spec/packet_io.h b/include/odp/api/spec/packet_io.h
index cec1f22..8802089 100644
--- a/include/odp/api/spec/packet_io.h
+++ b/include/odp/api/spec/packet_io.h
@@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {
 	 * interface capability before enabling the same. */
 	odp_bool_t enable_loop;
 
+	/** Inbound IPSEC inlined with packet input
+	 *
+	 *  Enable/disable inline inbound IPSEC operation. When enabled packet
+	 *  input directs all IPSEC packets automatically to IPSEC inbound
+	 *  processing. IPSEC configuration is done through the IPSEC API.
+	 *  Packets that are not (recognized as) IPSEC are processed
+	 *  according to the packet input configuration.
+	 *
+	 *  0: Disable inbound IPSEC inline operation (default)
+	 *  1: Enable inbound IPSEC inline operation
+	 *
+	 *  @see odp_ipsec_config(), odp_ipsec_sa_create()
+	 */
+	odp_bool_t inbound_ipsec;
+
+	/** Outbound IPSEC inlined with packet output
+	 *
+	 *  Enable/disable inline outbound IPSEC operation. When enabled IPSEC
+	 *  outbound processing can send outgoing IPSEC packets directly
+	 *  to the pktio interface for output. IPSEC configuration is done
+	 *  through the IPSEC API.
+	 *
+	 *  Outbound IPSEC inline operation cannot be combined with traffic
+	 *  manager (ODP_PKTOUT_MODE_TM).
+	 *
+	 *  0: Disable outbound IPSEC inline operation (default)
+	 *  1: Enable outbound IPSEC inline operation
+	 *
+	 *  @see odp_ipsec_config(), odp_ipsec_sa_create()
+	 */
+	odp_bool_t outbound_ipsec;
+
 } odp_pktio_config_t;
 
 /**

> >

> >  /**

> > @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

> >         ODP_IPSEC_LOOKUP_DISABLED = 0,

> >

> >         /** Inbound SA lookup is enabled. Used SPI values must be

> unique. */

> > -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> > +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> > +

> > +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> > +         * destination IP address. Used SPI values must be unique. */

> > +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

> >

> >  } odp_ipsec_lookup_mode_t;

> 

> odp_ipsec_lookup_mode_t is not added either in odp_ipsec_config() or

> in odp_ipsec_capability().

> I believe this should be added in both these struct?



This is part of odp_ipsec_sa_param_t, so lookup config is per SA.


> 

> >

> >

> >  /**

> > + * Result event destination

> > + */

> > +typedef enum odp_ipsec_dest_mode_t {

> > +       /** Destination for IPSEC result events is a queue. */

> > +       ODP_IPSEC_DEST_QUEUE = 0,

> > +

> > +       /** Destination for IPSEC result events is the classifier.

> > +        *  IPSEC capability 'cls_inline' determines if inline

> classification

> > +        *  is supported. */

> > +       ODP_IPSEC_DEST_CLS

> > +

> > +} odp_ipsec_dest_mode_t;

> 

> Should'nt we add a dest_mode ODP_IPSEC_DEST_PKTIO for outbound inline

> when the packet is sent out through interface directly.


This selection is for result events. For output direction, queue are the only option (for events). Queue vs inline pktout is selected by odp_ipsec_out_enq() vs odp_ipsec_out_inline(). Selection of output pktio (or TM queue in the future) is parameters to odp_ipsec_out_inline().

-Petri

Regards,
Bala


On 23 March 2017 at 17:32, Nikhil Agarwal <nikhil.agarwal@nxp.com> wrote:
> Few initial comments:

>

> - For Mixing inline and async processing must maintain same SA context in HW and implementation. In that case, we should shift to async SA creation so that while creating an SA, HW    interface can return the context to sw and Async processing can be done for that context.

> - Why we want to add different level of header retention, we can add retain header or discard.

> - Why parsing is optional and layer wise. Parsing should be same as on the pktio interface. Even in the lookaside APIs L3 parsing was mandatory. Otherwise application will have to do the parsing on its own. Moreover, application will never know L3 protocol if there are no parse results.

> - In case outer headers are retained in the packet, hardware will parse packet based outer headers not inner. Same goes for classification.

> - Checksum configuration (For inbound as well as outbound) can be per flow basis, as requirement can be different for different flows.

> - Support for ESP/AH and Tunnel/Transport mode should be exposed via capability.

> - Inline SA lookup should also support 3 tuple lookups (SIP, DIP, SPI).

> - Why there are separate lookup prams in SA, when SA tunnel params already include SIP and DIP?

> - What about enabling HASH distribution post IPSEC for FAT tunnel use case.

> - In per SA params, if dest_mode is ODP_IPSEC_DEST_CLS, why output is to a new CoS, why not to default CoS of that pktio. Are we saying that we will have two level of classification on that oktio, one before IPsec and one after that?


I am not sure if using the same default CoS of pktio would be
feasible, since it is possible that the default CoS could have L2 PMR
rules and the output packet is an IP packet. Do you have an issue in
implementing with a different CoS?

> - Who holds the responsibility of freeing the l2hdr in odp_ipsec_inline_op_param_t post transmission? IUs it mandated to be inside the packet? The why not just use l2_offset?

>

> Regards

> Nikhil

>

> -----Original Message-----

> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Petri Savolainen

> Sent: Tuesday, March 21, 2017 7:47 PM

> To: lng-odp@lists.linaro.org

> Subject: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

>

> Added support for inline IPSEC processing on packet input and output. Inline mode IPSEC and traffic manager cannot be enabled

> (currently) on the same pktio interface.

>

> Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

> ---

>  include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++++++++---

>  include/odp/api/spec/packet_io.h |  32 ++++

>  2 files changed, 355 insertions(+), 25 deletions(-)

>

> diff --git a/include/odp/api/spec/ipsec.h b/include/odp/api/spec/ipsec.h index e951e49..23d02cf 100644

> --- a/include/odp/api/spec/ipsec.h

> +++ b/include/odp/api/spec/ipsec.h

> @@ -19,6 +19,8 @@ extern "C" {

>  #endif

>

>  #include <odp/api/crypto.h>

> +#include <odp/api/packet_io.h>

> +#include <odp/api/classification.h>

>

>  /** @defgroup odp_ipsec ODP IPSEC

>   *  Operations of IPSEC API.

> @@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {

>           * Application uses asynchronous IPSEC operations,

>           * which return results via events.

>           */

> -       ODP_IPSEC_OP_MODE_ASYNC

> +       ODP_IPSEC_OP_MODE_ASYNC,

> +

> +       /** Inline IPSEC operation

> +         *

> +         * Packet input/output is connected directly to IPSEC inbound/outbound

> +         * processing. Application uses asynchronous or inline IPSEC

> +         * operations.

> +         */

> +       ODP_IPSEC_OP_MODE_INLINE,

> +

> +       /** IPSEC is disabled in inbound / outbound direction */

> +       ODP_IPSEC_OP_MODE_DISABLED

>

>  } odp_ipsec_op_mode_t;

>

>  /**

> + * Protocol layers in IPSEC configuration  */ typedef enum

> +odp_ipsec_proto_layer_t {

> +       /** No layers */

> +       ODP_IPSEC_LAYER_NONE = 0,

> +

> +       /** Layer L2 protocols (Ethernet, VLAN, etc) */

> +       ODP_IPSEC_LAYER_L2,

> +

> +       /** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

> +       ODP_IPSEC_LAYER_L3,

> +

> +       /** Layer L4 protocols (UDP, TCP, SCTP) */

> +       ODP_IPSEC_LAYER_L4,

> +

> +       /** All layers */

> +       ODP_IPSEC_LAYER_ALL

> +

> +} odp_ipsec_proto_layer_t;

> +

> +/**

>   * Configuration options for IPSEC inbound processing

>   */

>  typedef struct odp_ipsec_inbound_config_t { @@ -77,9 +111,113 @@ typedef struct odp_ipsec_inbound_config_t {

>                 uint32_t max;

>         } spi;

>

> +       /** Retain outer headers

> +        *

> +        *  Select up to which protocol layer (at least) outer headers are

> +        *  retained in inbound inline processing. Default value is

> +        *  ODP_IPSEC_LAYER_NONE.

> +        *

> +        *  ODP_IPSEC_LAYER_NONE: Application does not require any outer

> +        *                        headers to be retained.

> +        *

> +        *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.

> +        *

> +        *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3, otherwise the

> +        *                        same as ODP_IPSEC_LAYER_ALL.

> +        *

> +        *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4, otherwise the

> +        *                        same as ODP_IPSEC_LAYER_ALL.

> +        *

> +        *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before IPSEC are

> +        *                        retained. In transport mode, all headers

> +        *                        before IP (carrying IPSEC) are retained.

> +        *

> +        */

> +       odp_ipsec_proto_layer_t retain_outer;

> +

> +       /** Parse packet headers in IPSEC payload

> +        *

> +        *  Select header parsing level after inbound processing. Packet headers

> +        *  in IPSEC payload must be parsed (at least) up to this level.

> +        *  Default value is ODP_IPSEC_LAYER_NONE.

> +        *

> +        *  Note that IPSec payload is never a L2 packet (ODP_IPSEC_LAYER_L2

> +        *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC payload

> +        *  starts after IP header (ODP_IPSEC_LAYER_L3 equals

> +        *  ODP_IPSEC_LAYER_NONE).

> +        */

> +       odp_ipsec_proto_layer_t parse;

> +

> +       /** Flags to control IPSEC payload data checks up to the selected parse

> +        *  level. */

> +       union {

> +               struct {

> +                       /** Check IPv4 header checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t ipv4_chksum   : 1;

> +

> +                       /** Check UDP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t udp_chksum    : 1;

> +

> +                       /** Check TCP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t tcp_chksum    : 1;

> +

> +                       /** Check SCTP checksum in IPSEC payload.

> +                        *  Default value is 0. */

> +                       uint32_t sctp_chksum   : 1;

> +               } check;

> +

> +               /** All bits of the bit field structure

> +                 *

> +                 * This field can be used to set/clear all flags, or bitwise

> +                 * operations over the entire structure. */

> +               uint32_t all_check;

> +       };

> +

>  } odp_ipsec_inbound_config_t;

>

>  /**

> + * Configuration options for IPSEC outbound processing  */ typedef

> +struct odp_ipsec_outbound_config_t {

> +       /** Flags to control L3/L4 checksum insertion as part of outbound

> +        *  packet processing. Packet must have set with valid L3/L4 offsets.

> +        *  Checksum configuration is ignored for packets that checksum cannot

> +        *  be computed for (e.g. IPv4 fragments). Application may use a packet

> +        *  metadata flag to disable checksum insertion per packet bases.

> +        */

> +       union {

> +               struct {

> +                       /** Insert IPv4 header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_ipv4   : 1;

> +

> +                       /** Insert UDP header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_udp    : 1;

> +

> +                       /** Insert TCP header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_tcp    : 1;

> +

> +                       /** Insert SCTP header checksum on the payload packet

> +                        *  before IPSEC transformation. Default value is 0. */

> +                       uint32_t inner_sctp   : 1;

> +

> +               } chksum;

> +

> +               /** All bits of the bit field structure

> +                 *

> +                 * This field can be used to set/clear all flags, or bitwise

> +                 * operations over the entire structure. */

> +               uint32_t all_chksum;

> +       };

> +

> +} odp_ipsec_outbound_config_t;

> +

> +/**

>   * IPSEC capability

>   */

>  typedef struct odp_ipsec_capability_t { @@ -102,6 +240,24 @@ typedef struct odp_ipsec_capability_t {

>          */

>         uint8_t op_mode_async;

>

> +       /** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE) support

> +        *

> +        *  0: Inline IPSEC operation is not supported

> +        *  1: Inline IPSEC operation is supported

> +        *  2: Inline IPSEC operation is supported and preferred

> +        */

> +       uint8_t op_mode_inline;

> +

> +       /** Support of inline classification (ODP_IPSEC_DEST_CLS) for resulting

> +        *  inbound packets.

> +        *

> +        *  0: Inline classification of resulting packets is not supported

> +        *  1: Inline classification of resulting packets is supported

> +        *  2: Inline classification of resulting packets is supported and

> +        *     preferred

> +        */

> +       uint8_t cls_inline;

> +

>         /** Soft expiry limit in seconds support

>          *

>          *  0: Limit is not supported

> @@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {

>   * IPSEC configuration options

>   */

>  typedef struct odp_ipsec_config_t {

> -       /** IPSEC operation mode. Application selects which mode (sync or async)

> -        *  will be used for IPSEC operations.

> +       /** Inbound IPSEC operation mode. Application selects which mode

> +        *  will be used for inbound IPSEC operations.

>          *

>          *  @see odp_ipsec_in(), odp_ipsec_in_enq()

>          */

> -       odp_ipsec_op_mode_t op_mode;

> +       odp_ipsec_op_mode_t inbound_mode;

> +

> +       /** Outbound IPSEC operation mode. Application selects which mode

> +        *  will be used for outbound IPSEC operations.

> +        *

> +        *  @see odp_ipsec_out(), odp_ipsec_out_enq(), odp_ipsec_out_inline()

> +        */

> +       odp_ipsec_op_mode_t outbound_mode;

>

>         /** Maximum number of IPSEC SAs that application will use

>          * simultaneously */

> @@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {

>         /** IPSEC inbound processing configuration */

>         odp_ipsec_inbound_config_t inbound;

>

> +       /** IPSEC outbound processing configuration */

> +       odp_ipsec_outbound_config_t outbound;

> +

>  } odp_ipsec_config_t;

>

>  /**

> @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

>         ODP_IPSEC_LOOKUP_DISABLED = 0,

>

>         /** Inbound SA lookup is enabled. Used SPI values must be unique. */

> -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> +

> +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> +         * destination IP address. Used SPI values must be unique. */

> +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

>

>  } odp_ipsec_lookup_mode_t;

>

>  /**

> + * Result event destination

> + */

> +typedef enum odp_ipsec_dest_mode_t {

> +       /** Destination for IPSEC result events is a queue. */

> +       ODP_IPSEC_DEST_QUEUE = 0,

> +

> +       /** Destination for IPSEC result events is the classifier.

> +        *  IPSEC capability 'cls_inline' determines if inline classification

> +        *  is supported. */

> +       ODP_IPSEC_DEST_CLS

> +

> +} odp_ipsec_dest_mode_t;

> +

> +/**

>   * IPSEC Security Association (SA) parameters

>   */

>  typedef struct odp_ipsec_sa_param_t {

> @@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {

>         /** SPI value */

>         uint32_t spi;

>

> +       /** Additional inbound SA lookup parameters. Values are considered

> +        *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */

> +       struct {

> +               /* v4 or v6 */

> +               uint8_t ip_version;

> +

> +               /* IP destination address (NETWORK ENDIAN) */

> +               void    *dst_addr;

> +

> +       } lookup_param;

> +

>         /** MTU for outbound IP fragmentation offload

>          *

>          *  This is the maximum length of IP packets that outbound IPSEC @@ -434,13 +629,32 @@ typedef struct odp_ipsec_sa_param_t {

>          */

>         uint32_t mtu;

>

> +       /** Select where IPSEC result events are sent

> +        *

> +        *  Asynchronous and inline modes generate result events. Select where

> +        *  those events are sent. Inbound SAs may choose between a queue or

> +        *  the classifier. Outbound SAs must define a queue always.

> +        *  The default value is ODP_IPSEC_DEST_QUEUE.

> +        */

> +       odp_ipsec_dest_mode_t dest_mode;

> +

>         /** Destination queue for IPSEC events

>          *

> -        *  Operations in asynchronous mode enqueue resulting events into

> -        *  this queue.

> +        *  Operations in asynchronous or inline mode enqueue resulting events

> +        *  into this queue.

>          */

>         odp_queue_t dest_queue;

>

> +       /** Classifier destination CoS for IPSEC result events

> +        *

> +        *  Result events for successfully decapsulated packets are sent to

> +        *  classification through this CoS. Other result events are sent to

> +        *  'dest_queue'. This field is considered only when 'dest_mode' is

> +        *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any pktio

> +        *  interface default CoS.

> +        */

> +       odp_cos_t dest_cos;

> +

>         /** User defined SA context pointer

>          *

>          *  User defined context pointer associated with the SA.

> @@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {

>                 uint32_t all_error;

>         };

>

> +       union {

> +               /** Status flags */

> +               struct {

> +                       /** Packet was processed in inline mode */

> +                       uint32_t inline_mode      : 1;

> +

> +               } flag;

> +

> +               /** All flag bits */

> +               uint32_t all_flag;

> +       };

> +

>  } odp_ipsec_op_status_t;

>

>  /**

> @@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {  } odp_ipsec_op_param_t;

>

>  /**

> + * Outbound inline IPSEC operation parameters  */ typedef struct

> +odp_ipsec_inline_op_param_t {

> +       /** Packet output interface for inline output operation

> +        *

> +        *  Outbound inline IPSEC operation uses this packet IO interface to

> +        *  output the packet after a successful IPSEC transformation. The pktio

> +        *  must have been configured to operate in inline IPSEC mode.

> +        */

> +       odp_pktio_t pktio;

> +

> +       /** Outer headers for inline output operation

> +        *

> +        *  Outbound inline IPSEC operation uses this information to prepend

> +        *  outer headers to the IPSEC packet before sending it out.

> +        */

> +       struct {

> +               /** Points to first byte of outer headers to be copied in

> +                *  front of the outgoing IPSEC packet. Implementation copies

> +                *  the headers during odp_ipsec_out_inline() call. */

> +               uint8_t *ptr;

> +

> +               /** Outer header length in bytes */

> +               uint32_t len;

> +       } outer_hdr;

> +

> +} odp_ipsec_inline_op_param_t;

> +

> +/**

>   * IPSEC operation result for a packet

>   */

>  typedef struct odp_ipsec_packet_result_t { @@ -752,6 +1007,23 @@ typedef struct odp_ipsec_packet_result_t {

>          */

>         odp_ipsec_sa_t sa;

>

> +       /** Packet outer header status before inbound inline processing.

> +        *  This is valid only when status.flag.inline_mode is set.

> +        */

> +       struct {

> +               /** Points to the first byte of retained outer headers. These

> +                *  headers are stored in a contiquous, per packet,

> +                *  implementation specific memory space. Since the memory space

> +                *  may overlap with e.g. packet head/tailroom, the content

> +                *  becomes invalid if packet data storage is modified in

> +                *  anyway. The memory space may not be sharable to other

> +                *  threads. */

> +               uint8_t *ptr;

> +

> +               /** Outer header length in bytes */

> +               uint32_t len;

> +       } outer_hdr;

> +

>  } odp_ipsec_packet_result_t;

>

>  /**

> @@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {

>          *  at least 'num_pkt' elements.

>          *

>          *  Each successfully transformed packet has a valid value for these

> -        *  meta-data:

> +        *  meta-data regardless of the inner packet parse configuration.

> +        *  (odp_ipsec_inbound_config_t):

>          *  * L3 offset: Offset to the first byte of the (outmost) IP header

> -        *  * L4 offset: Offset to the first byte of the valid and known L4

> -        *               header (immediately following the IP header).

> -        *  * Various flags about L3 and L4 layers:

> -        *               has_l3, has_l4, has_ipv4, has_ipv6, has_ipfrag,

> -        *               has_ipsec, has_udp, has_tcp, etc depending on

> -        *               the resulted packet format

> +        *  * pktio:     For inbound inline IPSEC processed packets, original

> +        *               packet input interface

>          *

> -        * @see odp_packet_l3_offset(), odp_packet_l4_offset(),

> -        *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),

> -        *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()

> +        *  Other meta-data for parse results and error checks depend on

> +        *  configuration (selected parse and error check levels).

>          */

>         odp_packet_t *pkt;

>

> @@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>  /**

>   * Inbound asynchronous IPSEC operation

>   *

> - * This operation does inbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to

> - * odp_ipsec_in(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.

> + * This operation does inbound IPSEC processing in asynchronous mode.

> + It

> + * processes packets otherwise identically to odp_ipsec_in(), but

> + outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> + following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler context @@ -928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>   * events for the same SA are enqueued in order, and packet handles (for the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for

> + IPSEC

> + * packets for which inline processing is not possible. Packets for the

> + same SA

> + * may be processed simultaneously in both modes (initiated by this

> + function

> + * and inline operation).

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt) @@ -940,10 +1213,10 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>  /**

>   * Outbound asynchronous IPSEC operation

>   *

> - * This operation does outbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to

> - * odp_ipsec_out(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.

> + * This operation does outbound IPSEC processing in asynchronous mode.

> + It

> + * processes packets otherwise identically to odp_ipsec_out(), but

> + outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> + following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler context @@ -953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>   * events for the same SA are enqueued in order, and packet handles (for the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for

> + IPSEC

> + * packets for which inline processing is not possible.

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt) @@ -963,6 +1239,28 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);  int odp_ipsec_out_enq(const odp_ipsec_op_param_t *input);

>

>  /**

> + * Outbound inline IPSEC operation

> + *

> + * This operation does outbound inline IPSEC processing for the

> +packets. It's

> + * otherwise identical to odp_ipsec_out_enq(), but outputs all

> +successfully

> + * transformed packets to the specified output interface, instead of

> +generating

> + * result events for those.

> + *

> + * Inline operation parameters are defined per packet. The array of

> +parameters

> + * must have 'op_param.num_pkt' elements and is pointed to by 'inline_param'.

> + *

> + * @param         op_param      Operation parameters

> + * @param         inline_param  Outbound inline operation specific parameters

> + *

> + * @return Number of packets consumed (0 ... op_param.num_pkt)

> + * @retval <0     On failure

> + *

> + * @see odp_ipsec_out_enq()

> + */

> +int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,

> +                        const odp_ipsec_inline_op_param_t *inline_param);

> +

> +/**

>   * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event

>   *

>   * Copies IPSEC operation results from an event. The event must be of diff --git a/include/odp/api/spec/packet_io.h b/include/odp/api/spec/packet_io.h

> index cec1f22..8802089 100644

> --- a/include/odp/api/spec/packet_io.h

> +++ b/include/odp/api/spec/packet_io.h

> @@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {

>          * interface capability before enabling the same. */

>         odp_bool_t enable_loop;

>

> +       /** Inbound IPSEC inlined with packet input

> +        *

> +        *  Enable/disable inline inbound IPSEC operation. When enabled packet

> +        *  input directs all IPSEC packets automatically to IPSEC inbound

> +        *  processing. IPSEC configuration is done through the IPSEC API.

> +        *  Packets that are not (recognized as) IPSEC are processed

> +        *  according to the packet input configuration.

> +        *

> +        *  0: Disable inbound IPSEC inline operation (default)

> +        *  1: Enable inbound IPSEC inline operation

> +        *

> +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +        */

> +       odp_bool_t inbound_ipsec;

> +

> +       /** Outbound IPSEC inlined with packet output

> +        *

> +        *  Enable/disable inline outbound IPSEC operation. When enabled IPSEC

> +        *  outbound processing can send outgoing IPSEC packets directly

> +        *  to the pktio interface for output. IPSEC configuration is done

> +        *  through the IPSEC API.

> +        *

> +        *  Outbound IPSEC inline operation cannot be combined with traffic

> +        *  manager (ODP_PKTOUT_MODE_TM).

> +        *

> +        *  0: Disable outbound IPSEC inline operation (default)

> +        *  1: Enable outbound IPSEC inline operation

> +        *

> +        *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +        */

> +       odp_bool_t outbound_ipsec;

> +

>  } odp_pktio_config_t;

>

>  /**

> --

> 2.8.1

>

Regards,
Bala


On 23 March 2017 at 17:40, Savolainen, Petri (Nokia - FI/Espoo)
<petri.savolainen@nokia-bell-labs.com> wrote:
>> >

>> >  /**

>> > @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

>> >         ODP_IPSEC_LOOKUP_DISABLED = 0,

>> >

>> >         /** Inbound SA lookup is enabled. Used SPI values must be

>> unique. */

>> > -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

>> > +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

>> > +

>> > +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

>> > +         * destination IP address. Used SPI values must be unique. */

>> > +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

>> >

>> >  } odp_ipsec_lookup_mode_t;

>>

>> odp_ipsec_lookup_mode_t is not added either in odp_ipsec_config() or

>> in odp_ipsec_capability().

>> I believe this should be added in both these struct?

>

>

> This is part of odp_ipsec_sa_param_t, so lookup config is per SA.


Does that mean that both SPI lookup and DEST ADDR lookup are mandatory?
Is there a reason lookup mode is per SA?

>

>

>>

>> >

>> >

>> >  /**

>> > + * Result event destination

>> > + */

>> > +typedef enum odp_ipsec_dest_mode_t {

>> > +       /** Destination for IPSEC result events is a queue. */

>> > +       ODP_IPSEC_DEST_QUEUE = 0,

>> > +

>> > +       /** Destination for IPSEC result events is the classifier.

>> > +        *  IPSEC capability 'cls_inline' determines if inline

>> classification

>> > +        *  is supported. */

>> > +       ODP_IPSEC_DEST_CLS

>> > +

>> > +} odp_ipsec_dest_mode_t;

>>

>> Should'nt we add a dest_mode ODP_IPSEC_DEST_PKTIO for outbound inline

>> when the packet is sent out through interface directly.

>

> This selection is for result events. For output direction, queue are the only option (for events). Queue vs inline pktout is selected by odp_ipsec_out_enq() vs odp_ipsec_out_inline(). Selection of output pktio (or TM queue in the future) is parameters to odp_ipsec_out_inline().


Yes. But the odp_ipsec_dest_mode_t is available in SA params and if
the SA is configured in outbound direction and linked to the pktio
then the configuration of dest_mode cannot be ODP_IPSEC_DEST_QUEUE.

>

> -Petri

>

>

>

Hi,

A few quick comments below. Petri will probably comment the other points.

	Janne

> -----Original Message-----

> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Nikhil Agarwal

> Sent: Thursday, March 23, 2017 2:02 PM

> To: Petri Savolainen <petri.savolainen@linaro.org>; lng-odp@lists.linaro.org

> Subject: Re: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

> 

> Few initial comments:

> 

> - For Mixing inline and async processing must maintain same SA context in HW and

> implementation. In that case, we should shift to async SA creation so that while creating

> an SA, HW    interface can return the context to sw and Async processing can be done for

> that context.

> - Why we want to add different level of header retention, we can add retain header or

> discard.

> - Why parsing is optional and layer wise. Parsing should be same as on the pktio

> interface. Even in the lookaside APIs L3 parsing was mandatory. Otherwise application will

> have to do the parsing on its own. Moreover, application will never know L3 protocol if

> there are no parse results.

> - In case outer headers are retained in the packet, hardware will parse packet based outer

> headers not inner. Same goes for classification.


Outer headers are not provided _in the packet_. The retained outer headers are not
part of the packet data but stored with the packet in some implementation defined
way.

Therefore parsing and classification is done based on the inner headers (or rather,
the packet as it appears after IPsec decapsulation).

> - Checksum configuration (For inbound as well as outbound) can be per flow basis, as

> requirement can be different for different flows.


It depends on the definition of flow here. Both locally generated and forwarded traffic
may be sent out through the same outbound SA. For forwarded traffic L4 checksums should
not be (re)generated but for locally originated traffic they should. Also, some UDP
protocols like VxLAN do not use checksum.

So, I think checksum generation should be selectable per packet. This is not part
of the latest IPsec patch since it probably needs to be done in the generic
packet API. In fact, packet_io.h currently says: "Application may use a packet
metadata flag to disable checksum insertion per packet bases." Such feature is
just missing.

> - Support for ESP/AH and Tunnel/Transport mode should be exposed via capability.

> - Inline SA lookup should also support 3 tuple lookups (SIP, DIP, SPI).


Now lookups that would need source IP would need to be done by the application
in SW. Support in the inline IPsec API can be added later, but I am not sure
if it is needed there right away (it would only be needed for multicast IPsec
traffic anyway, right?).

Right now the API assumes that inbound SAs have unique SPIs, which can make
things easier to implementations. With source address matching in addition
to destination address and SPI you would presumably have non-unique SPIs too.

> - Why there are separate lookup prams in SA, when SA tunnel params already include SIP and

> DIP?


Transport mode SAs do not have the IP addresses.

> - What about enabling HASH distribution post IPSEC for FAT tunnel use case.

> - In per SA params, if dest_mode is ODP_IPSEC_DEST_CLS, why output is to a new CoS, why

> not to default CoS of that pktio. Are we saying that we will have two level of

> classification on that oktio, one before IPsec and one after that?


One reason is that separate CoS allows one to distinguish IPsec
processed packets from other packets in the classification rules.

> - Who holds the responsibility of freeing the l2hdr in odp_ipsec_inline_op_param_t post

> transmission? IUs it mandated to be inside the packet? The why not just use l2_offset?


The implementation copies the l2 header data to its internal storage during
the enqueue call. ODP implementation will not dereference the provided pointer
after the enqueue call returns, so the application is then free to do anything
with it. 
 
> 

> Regards

> Nikhil

> 

> -----Original Message-----

> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Petri Savolainen

> Sent: Tuesday, March 21, 2017 7:47 PM

> To: lng-odp@lists.linaro.org

> Subject: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

> 

> Added support for inline IPSEC processing on packet input and output. Inline mode IPSEC

> and traffic manager cannot be enabled

> (currently) on the same pktio interface.

> 

> Signed-off-by: Petri Savolainen <petri.savolainen@linaro.org>

> ---

>  include/odp/api/spec/ipsec.h     | 348 ++++++++++++++++++++++++++++++++++++---

>  include/odp/api/spec/packet_io.h |  32 ++++

>  2 files changed, 355 insertions(+), 25 deletions(-)

> 

> diff --git a/include/odp/api/spec/ipsec.h b/include/odp/api/spec/ipsec.h index

> e951e49..23d02cf 100644

> --- a/include/odp/api/spec/ipsec.h

> +++ b/include/odp/api/spec/ipsec.h

> @@ -19,6 +19,8 @@ extern "C" {

>  #endif

> 

>  #include <odp/api/crypto.h>

> +#include <odp/api/packet_io.h>

> +#include <odp/api/classification.h>

> 

>  /** @defgroup odp_ipsec ODP IPSEC

>   *  Operations of IPSEC API.

> @@ -51,11 +53,43 @@ typedef enum odp_ipsec_op_mode_t {

>  	  * Application uses asynchronous IPSEC operations,

>  	  * which return results via events.

>  	  */

> -	ODP_IPSEC_OP_MODE_ASYNC

> +	ODP_IPSEC_OP_MODE_ASYNC,

> +

> +	/** Inline IPSEC operation

> +	  *

> +	  * Packet input/output is connected directly to IPSEC inbound/outbound

> +	  * processing. Application uses asynchronous or inline IPSEC

> +	  * operations.

> +	  */

> +	ODP_IPSEC_OP_MODE_INLINE,

> +

> +	/** IPSEC is disabled in inbound / outbound direction */

> +	ODP_IPSEC_OP_MODE_DISABLED

> 

>  } odp_ipsec_op_mode_t;

> 

>  /**

> + * Protocol layers in IPSEC configuration  */ typedef enum

> +odp_ipsec_proto_layer_t {

> +	/** No layers */

> +	ODP_IPSEC_LAYER_NONE = 0,

> +

> +	/** Layer L2 protocols (Ethernet, VLAN, etc) */

> +	ODP_IPSEC_LAYER_L2,

> +

> +	/** Layer L3 protocols (IPv4, IPv6, ICMP, IPSec, etc) */

> +	ODP_IPSEC_LAYER_L3,

> +

> +	/** Layer L4 protocols (UDP, TCP, SCTP) */

> +	ODP_IPSEC_LAYER_L4,

> +

> +	/** All layers */

> +	ODP_IPSEC_LAYER_ALL

> +

> +} odp_ipsec_proto_layer_t;

> +

> +/**

>   * Configuration options for IPSEC inbound processing

>   */

>  typedef struct odp_ipsec_inbound_config_t { @@ -77,9 +111,113 @@ typedef struct

> odp_ipsec_inbound_config_t {

>  		uint32_t max;

>  	} spi;

> 

> +	/** Retain outer headers

> +	 *

> +	 *  Select up to which protocol layer (at least) outer headers are

> +	 *  retained in inbound inline processing. Default value is

> +	 *  ODP_IPSEC_LAYER_NONE.

> +	 *

> +	 *  ODP_IPSEC_LAYER_NONE: Application does not require any outer

> +	 *                        headers to be retained.

> +	 *

> +	 *  ODP_IPSEC_LAYER_L2:   Retain headers up to layer 2.

> +	 *

> +	 *  ODP_IPSEC_LAYER_L3:   Retain headers up to layer 3, otherwise the

> +	 *                        same as ODP_IPSEC_LAYER_ALL.

> +	 *

> +	 *  ODP_IPSEC_LAYER_L4:   Retain headers up to layer 4, otherwise the

> +	 *                        same as ODP_IPSEC_LAYER_ALL.

> +	 *

> +	 *  ODP_IPSEC_LAYER_ALL:  In tunnel mode, all headers before IPSEC are

> +	 *                        retained. In transport mode, all headers

> +	 *                        before IP (carrying IPSEC) are retained.

> +	 *

> +	 */

> +	odp_ipsec_proto_layer_t retain_outer;

> +

> +	/** Parse packet headers in IPSEC payload

> +	 *

> +	 *  Select header parsing level after inbound processing. Packet headers

> +	 *  in IPSEC payload must be parsed (at least) up to this level.

> +	 *  Default value is ODP_IPSEC_LAYER_NONE.

> +	 *

> +	 *  Note that IPSec payload is never a L2 packet (ODP_IPSEC_LAYER_L2

> +	 *  equals ODP_IPSEC_LAYER_NONE). In transport mode, IPSEC payload

> +	 *  starts after IP header (ODP_IPSEC_LAYER_L3 equals

> +	 *  ODP_IPSEC_LAYER_NONE).

> +	 */

> +	odp_ipsec_proto_layer_t parse;

> +

> +	/** Flags to control IPSEC payload data checks up to the selected parse

> +	 *  level. */

> +	union {

> +		struct {

> +			/** Check IPv4 header checksum in IPSEC payload.

> +			 *  Default value is 0. */

> +			uint32_t ipv4_chksum   : 1;

> +

> +			/** Check UDP checksum in IPSEC payload.

> +			 *  Default value is 0. */

> +			uint32_t udp_chksum    : 1;

> +

> +			/** Check TCP checksum in IPSEC payload.

> +			 *  Default value is 0. */

> +			uint32_t tcp_chksum    : 1;

> +

> +			/** Check SCTP checksum in IPSEC payload.

> +			 *  Default value is 0. */

> +			uint32_t sctp_chksum   : 1;

> +		} check;

> +

> +		/** All bits of the bit field structure

> +		  *

> +		  * This field can be used to set/clear all flags, or bitwise

> +		  * operations over the entire structure. */

> +		uint32_t all_check;

> +	};

> +

>  } odp_ipsec_inbound_config_t;

> 

>  /**

> + * Configuration options for IPSEC outbound processing  */ typedef

> +struct odp_ipsec_outbound_config_t {

> +	/** Flags to control L3/L4 checksum insertion as part of outbound

> +	 *  packet processing. Packet must have set with valid L3/L4 offsets.

> +	 *  Checksum configuration is ignored for packets that checksum cannot

> +	 *  be computed for (e.g. IPv4 fragments). Application may use a packet

> +	 *  metadata flag to disable checksum insertion per packet bases.

> +	 */

> +	union {

> +		struct {

> +			/** Insert IPv4 header checksum on the payload packet

> +			 *  before IPSEC transformation. Default value is 0. */

> +			uint32_t inner_ipv4   : 1;

> +

> +			/** Insert UDP header checksum on the payload packet

> +			 *  before IPSEC transformation. Default value is 0. */

> +			uint32_t inner_udp    : 1;

> +

> +			/** Insert TCP header checksum on the payload packet

> +			 *  before IPSEC transformation. Default value is 0. */

> +			uint32_t inner_tcp    : 1;

> +

> +			/** Insert SCTP header checksum on the payload packet

> +			 *  before IPSEC transformation. Default value is 0. */

> +			uint32_t inner_sctp   : 1;

> +

> +		} chksum;

> +

> +		/** All bits of the bit field structure

> +		  *

> +		  * This field can be used to set/clear all flags, or bitwise

> +		  * operations over the entire structure. */

> +		uint32_t all_chksum;

> +	};

> +

> +} odp_ipsec_outbound_config_t;

> +

> +/**

>   * IPSEC capability

>   */

>  typedef struct odp_ipsec_capability_t { @@ -102,6 +240,24 @@ typedef struct

> odp_ipsec_capability_t {

>  	 */

>  	uint8_t op_mode_async;

> 

> +	/** Inline IPSEC operation mode (ODP_IPSEC_OP_MODE_INLINE) support

> +	 *

> +	 *  0: Inline IPSEC operation is not supported

> +	 *  1: Inline IPSEC operation is supported

> +	 *  2: Inline IPSEC operation is supported and preferred

> +	 */

> +	uint8_t op_mode_inline;

> +

> +	/** Support of inline classification (ODP_IPSEC_DEST_CLS) for resulting

> +	 *  inbound packets.

> +	 *

> +	 *  0: Inline classification of resulting packets is not supported

> +	 *  1: Inline classification of resulting packets is supported

> +	 *  2: Inline classification of resulting packets is supported and

> +	 *     preferred

> +	 */

> +	uint8_t cls_inline;

> +

>  	/** Soft expiry limit in seconds support

>  	 *

>  	 *  0: Limit is not supported

> @@ -128,12 +284,19 @@ typedef struct odp_ipsec_capability_t {

>   * IPSEC configuration options

>   */

>  typedef struct odp_ipsec_config_t {

> -	/** IPSEC operation mode. Application selects which mode (sync or async)

> -	 *  will be used for IPSEC operations.

> +	/** Inbound IPSEC operation mode. Application selects which mode

> +	 *  will be used for inbound IPSEC operations.

>  	 *

>  	 *  @see odp_ipsec_in(), odp_ipsec_in_enq()

>  	 */

> -	odp_ipsec_op_mode_t op_mode;

> +	odp_ipsec_op_mode_t inbound_mode;

> +

> +	/** Outbound IPSEC operation mode. Application selects which mode

> +	 *  will be used for outbound IPSEC operations.

> +	 *

> +	 *  @see odp_ipsec_out(), odp_ipsec_out_enq(), odp_ipsec_out_inline()

> +	 */

> +	odp_ipsec_op_mode_t outbound_mode;

> 

>  	/** Maximum number of IPSEC SAs that application will use

>  	 * simultaneously */

> @@ -142,6 +305,9 @@ typedef struct odp_ipsec_config_t {

>  	/** IPSEC inbound processing configuration */

>  	odp_ipsec_inbound_config_t inbound;

> 

> +	/** IPSEC outbound processing configuration */

> +	odp_ipsec_outbound_config_t outbound;

> +

>  } odp_ipsec_config_t;

> 

>  /**

> @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

>  	ODP_IPSEC_LOOKUP_DISABLED = 0,

> 

>  	/** Inbound SA lookup is enabled. Used SPI values must be unique. */

> -	ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> +	ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> +

> +	/** Inbound SA lookup is enabled. Lookup matches both SPI and

> +	  * destination IP address. Used SPI values must be unique. */

> +	ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

> 

>  } odp_ipsec_lookup_mode_t;

> 

>  /**

> + * Result event destination

> + */

> +typedef enum odp_ipsec_dest_mode_t {

> +	/** Destination for IPSEC result events is a queue. */

> +	ODP_IPSEC_DEST_QUEUE = 0,

> +

> +	/** Destination for IPSEC result events is the classifier.

> +	 *  IPSEC capability 'cls_inline' determines if inline classification

> +	 *  is supported. */

> +	ODP_IPSEC_DEST_CLS

> +

> +} odp_ipsec_dest_mode_t;

> +

> +/**

>   * IPSEC Security Association (SA) parameters

>   */

>  typedef struct odp_ipsec_sa_param_t {

> @@ -426,6 +610,17 @@ typedef struct odp_ipsec_sa_param_t {

>  	/** SPI value */

>  	uint32_t spi;

> 

> +	/** Additional inbound SA lookup parameters. Values are considered

> +	 *  only in ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI lookup mode. */

> +	struct {

> +		/* v4 or v6 */

> +		uint8_t ip_version;

> +

> +		/* IP destination address (NETWORK ENDIAN) */

> +		void    *dst_addr;

> +

> +	} lookup_param;

> +

>  	/** MTU for outbound IP fragmentation offload

>  	 *

>  	 *  This is the maximum length of IP packets that outbound IPSEC @@ -434,13

> +629,32 @@ typedef struct odp_ipsec_sa_param_t {

>  	 */

>  	uint32_t mtu;

> 

> +	/** Select where IPSEC result events are sent

> +	 *

> +	 *  Asynchronous and inline modes generate result events. Select where

> +	 *  those events are sent. Inbound SAs may choose between a queue or

> +	 *  the classifier. Outbound SAs must define a queue always.

> +	 *  The default value is ODP_IPSEC_DEST_QUEUE.

> +	 */

> +	odp_ipsec_dest_mode_t dest_mode;

> +

>  	/** Destination queue for IPSEC events

>  	 *

> -	 *  Operations in asynchronous mode enqueue resulting events into

> -	 *  this queue.

> +	 *  Operations in asynchronous or inline mode enqueue resulting events

> +	 *  into this queue.

>  	 */

>  	odp_queue_t dest_queue;

> 

> +	/** Classifier destination CoS for IPSEC result events

> +	 *

> +	 *  Result events for successfully decapsulated packets are sent to

> +	 *  classification through this CoS. Other result events are sent to

> +	 *  'dest_queue'. This field is considered only when 'dest_mode' is

> +	 *  ODP_IPSEC_DEST_CLS. The CoS must not be shared between any pktio

> +	 *  interface default CoS.

> +	 */

> +	odp_cos_t dest_cos;

> +

>  	/** User defined SA context pointer

>  	 *

>  	 *  User defined context pointer associated with the SA.

> @@ -673,6 +887,18 @@ typedef struct odp_ipsec_op_status_t {

>  		uint32_t all_error;

>  	};

> 

> +	union {

> +		/** Status flags */

> +		struct {

> +			/** Packet was processed in inline mode */

> +			uint32_t inline_mode      : 1;

> +

> +		} flag;

> +

> +		/** All flag bits */

> +		uint32_t all_flag;

> +	};

> +

>  } odp_ipsec_op_status_t;

> 

>  /**

> @@ -727,6 +953,35 @@ typedef struct odp_ipsec_op_param_t {  } odp_ipsec_op_param_t;

> 

>  /**

> + * Outbound inline IPSEC operation parameters  */ typedef struct

> +odp_ipsec_inline_op_param_t {

> +	/** Packet output interface for inline output operation

> +	 *

> +	 *  Outbound inline IPSEC operation uses this packet IO interface to

> +	 *  output the packet after a successful IPSEC transformation. The pktio

> +	 *  must have been configured to operate in inline IPSEC mode.

> +	 */

> +	odp_pktio_t pktio;

> +

> +	/** Outer headers for inline output operation

> +	 *

> +	 *  Outbound inline IPSEC operation uses this information to prepend

> +	 *  outer headers to the IPSEC packet before sending it out.

> +	 */

> +	struct {

> +		/** Points to first byte of outer headers to be copied in

> +		 *  front of the outgoing IPSEC packet. Implementation copies

> +		 *  the headers during odp_ipsec_out_inline() call. */

> +		uint8_t *ptr;

> +

> +		/** Outer header length in bytes */

> +		uint32_t len;

> +	} outer_hdr;

> +

> +} odp_ipsec_inline_op_param_t;

> +

> +/**

>   * IPSEC operation result for a packet

>   */

>  typedef struct odp_ipsec_packet_result_t { @@ -752,6 +1007,23 @@ typedef struct

> odp_ipsec_packet_result_t {

>  	 */

>  	odp_ipsec_sa_t sa;

> 

> +	/** Packet outer header status before inbound inline processing.

> +	 *  This is valid only when status.flag.inline_mode is set.

> +	 */

> +	struct {

> +		/** Points to the first byte of retained outer headers. These

> +		 *  headers are stored in a contiquous, per packet,

> +		 *  implementation specific memory space. Since the memory space

> +		 *  may overlap with e.g. packet head/tailroom, the content

> +		 *  becomes invalid if packet data storage is modified in

> +		 *  anyway. The memory space may not be sharable to other

> +		 *  threads. */

> +		uint8_t *ptr;

> +

> +		/** Outer header length in bytes */

> +		uint32_t len;

> +	} outer_hdr;

> +

>  } odp_ipsec_packet_result_t;

> 

>  /**

> @@ -773,18 +1045,14 @@ typedef struct odp_ipsec_op_result_t {

>  	 *  at least 'num_pkt' elements.

>  	 *

>  	 *  Each successfully transformed packet has a valid value for these

> -	 *  meta-data:

> +	 *  meta-data regardless of the inner packet parse configuration.

> +	 *  (odp_ipsec_inbound_config_t):

>  	 *  * L3 offset: Offset to the first byte of the (outmost) IP header

> -	 *  * L4 offset: Offset to the first byte of the valid and known L4

> -	 *               header (immediately following the IP header).

> -	 *  * Various flags about L3 and L4 layers:

> -	 *               has_l3, has_l4, has_ipv4, has_ipv6, has_ipfrag,

> -	 *               has_ipsec, has_udp, has_tcp, etc depending on

> -	 *               the resulted packet format

> +	 *  * pktio:     For inbound inline IPSEC processed packets, original

> +	 *               packet input interface

>  	 *

> -	 * @see odp_packet_l3_offset(), odp_packet_l4_offset(),

> -	 *      odp_packet_has_ipv4(), odp_packet_has_ipv6(),

> -	 *      odp_packet_has_ipfrag(), odp_packet_has_ipsec()

> +	 *  Other meta-data for parse results and error checks depend on

> +	 *  configuration (selected parse and error check levels).

>  	 */

>  	odp_packet_t *pkt;

> 

> @@ -915,10 +1183,10 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>  /**

>   * Inbound asynchronous IPSEC operation

>   *

> - * This operation does inbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to

> - * odp_ipsec_in(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.

> + * This operation does inbound IPSEC processing in asynchronous mode.

> + It

> + * processes packets otherwise identically to odp_ipsec_in(), but

> + outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> + following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler context @@ -

> 928,6 +1196,11 @@ int odp_ipsec_out(const odp_ipsec_op_param_t *input,

>   * events for the same SA are enqueued in order, and packet handles (for the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for

> + IPSEC

> + * packets for which inline processing is not possible. Packets for the

> + same SA

> + * may be processed simultaneously in both modes (initiated by this

> + function

> + * and inline operation).

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt) @@ -940,10 +1213,10 @@

> int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>  /**

>   * Outbound asynchronous IPSEC operation

>   *

> - * This operation does outbound IPSEC processing in asynchronous mode

> - * (ODP_IPSEC_OP_MODE_ASYNC). It processes packets otherwise identically to

> - * odp_ipsec_out(), but outputs all results through one or more

> - * ODP_EVENT_IPSEC_RESULT events with the following ordering considerations.

> + * This operation does outbound IPSEC processing in asynchronous mode.

> + It

> + * processes packets otherwise identically to odp_ipsec_out(), but

> + outputs all

> + * results through one or more ODP_EVENT_IPSEC_RESULT events with the

> + following

> + * ordering considerations.

>   *

>   * Asynchronous mode maintains (operation input) packet order per SA when

>   * application calls the operation within an ordered or atomic scheduler context @@ -

> 953,6 +1226,9 @@ int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);

>   * events for the same SA are enqueued in order, and packet handles (for the

>   * same SA) are stored in order within an event.

>   *

> + * The function may be used also in inline processing mode, e.g. for

> + IPSEC

> + * packets for which inline processing is not possible.

> + *

>   * @param         input   Operation input parameters

>   *

>   * @return Number of input packets consumed (0 ... input.num_pkt) @@ -963,6 +1239,28 @@

> int odp_ipsec_in_enq(const odp_ipsec_op_param_t *input);  int odp_ipsec_out_enq(const

> odp_ipsec_op_param_t *input);

> 

>  /**

> + * Outbound inline IPSEC operation

> + *

> + * This operation does outbound inline IPSEC processing for the

> +packets. It's

> + * otherwise identical to odp_ipsec_out_enq(), but outputs all

> +successfully

> + * transformed packets to the specified output interface, instead of

> +generating

> + * result events for those.

> + *

> + * Inline operation parameters are defined per packet. The array of

> +parameters

> + * must have 'op_param.num_pkt' elements and is pointed to by 'inline_param'.

> + *

> + * @param         op_param      Operation parameters

> + * @param         inline_param  Outbound inline operation specific parameters

> + *

> + * @return Number of packets consumed (0 ... op_param.num_pkt)

> + * @retval <0     On failure

> + *

> + * @see odp_ipsec_out_enq()

> + */

> +int odp_ipsec_out_inline(const odp_ipsec_op_param_t *op_param,

> +			 const odp_ipsec_inline_op_param_t *inline_param);

> +

> +/**

>   * Get IPSEC results from an ODP_EVENT_IPSEC_RESULT event

>   *

>   * Copies IPSEC operation results from an event. The event must be of diff --git

> a/include/odp/api/spec/packet_io.h b/include/odp/api/spec/packet_io.h

> index cec1f22..8802089 100644

> --- a/include/odp/api/spec/packet_io.h

> +++ b/include/odp/api/spec/packet_io.h

> @@ -407,6 +407,38 @@ typedef struct odp_pktio_config_t {

>  	 * interface capability before enabling the same. */

>  	odp_bool_t enable_loop;

> 

> +	/** Inbound IPSEC inlined with packet input

> +	 *

> +	 *  Enable/disable inline inbound IPSEC operation. When enabled packet

> +	 *  input directs all IPSEC packets automatically to IPSEC inbound

> +	 *  processing. IPSEC configuration is done through the IPSEC API.

> +	 *  Packets that are not (recognized as) IPSEC are processed

> +	 *  according to the packet input configuration.

> +	 *

> +	 *  0: Disable inbound IPSEC inline operation (default)

> +	 *  1: Enable inbound IPSEC inline operation

> +	 *

> +	 *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +	 */

> +	odp_bool_t inbound_ipsec;

> +

> +	/** Outbound IPSEC inlined with packet output

> +	 *

> +	 *  Enable/disable inline outbound IPSEC operation. When enabled IPSEC

> +	 *  outbound processing can send outgoing IPSEC packets directly

> +	 *  to the pktio interface for output. IPSEC configuration is done

> +	 *  through the IPSEC API.

> +	 *

> +	 *  Outbound IPSEC inline operation cannot be combined with traffic

> +	 *  manager (ODP_PKTOUT_MODE_TM).

> +	 *

> +	 *  0: Disable outbound IPSEC inline operation (default)

> +	 *  1: Enable outbound IPSEC inline operation

> +	 *

> +	 *  @see odp_ipsec_config(), odp_ipsec_sa_create()

> +	 */

> +	odp_bool_t outbound_ipsec;

> +

>  } odp_pktio_config_t;

> 

>  /**

> --

> 2.8.1

> > This is part of odp_ipsec_sa_param_t, so lookup config is per SA.

> 

> Does that mean that both SPI lookup and DEST ADDR lookup are mandatory?

> Is there a reason lookup mode is per SA?


The lookup mode is per SA to make it possible to have SAs for which
not ODP but the application performs the lookup (e.g. currently for
multicast IPsec SAs that do not have unique SPIs and require src
address lookup too, or for any other SAs with weird lookup rules)
and uses look-a-side IPsec ops with an explicit SA to do the
IPsec transforms.

Thus, I think at minimum the ODP_IPSEC_LOOKUP_DISABLED needs to
be per-SA even if the SPI versus SPI+dstaddr selection would be
global.

	Janne


> -----Original Message-----

> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Bala Manoharan

> Sent: Thursday, March 23, 2017 4:42 PM

> To: Savolainen, Petri (Nokia - FI/Espoo) <petri.savolainen@nokia-bell-labs.com>

> Cc: lng-odp-forward <lng-odp@lists.linaro.org>

> Subject: Re: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

> 

> Regards,

> Bala

> 

> 

> On 23 March 2017 at 17:40, Savolainen, Petri (Nokia - FI/Espoo)

> <petri.savolainen@nokia-bell-labs.com> wrote:

> >> >

> >> >  /**

> >> > @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

> >> >         ODP_IPSEC_LOOKUP_DISABLED = 0,

> >> >

> >> >         /** Inbound SA lookup is enabled. Used SPI values must be

> >> unique. */

> >> > -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

> >> > +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

> >> > +

> >> > +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

> >> > +         * destination IP address. Used SPI values must be unique. */

> >> > +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

> >> >

> >> >  } odp_ipsec_lookup_mode_t;

> >>

> >> odp_ipsec_lookup_mode_t is not added either in odp_ipsec_config() or

> >> in odp_ipsec_capability().

> >> I believe this should be added in both these struct?

> >

> >

> > This is part of odp_ipsec_sa_param_t, so lookup config is per SA.

> 

> Does that mean that both SPI lookup and DEST ADDR lookup are mandatory?

> Is there a reason lookup mode is per SA?

> 

> >

> >

> >>

> >> >

> >> >

> >> >  /**

> >> > + * Result event destination

> >> > + */

> >> > +typedef enum odp_ipsec_dest_mode_t {

> >> > +       /** Destination for IPSEC result events is a queue. */

> >> > +       ODP_IPSEC_DEST_QUEUE = 0,

> >> > +

> >> > +       /** Destination for IPSEC result events is the classifier.

> >> > +        *  IPSEC capability 'cls_inline' determines if inline

> >> classification

> >> > +        *  is supported. */

> >> > +       ODP_IPSEC_DEST_CLS

> >> > +

> >> > +} odp_ipsec_dest_mode_t;

> >>

> >> Should'nt we add a dest_mode ODP_IPSEC_DEST_PKTIO for outbound inline

> >> when the packet is sent out through interface directly.

> >

> > This selection is for result events. For output direction, queue are the only option

> (for events). Queue vs inline pktout is selected by odp_ipsec_out_enq() vs

> odp_ipsec_out_inline(). Selection of output pktio (or TM queue in the future) is

> parameters to odp_ipsec_out_inline().

> 

> Yes. But the odp_ipsec_dest_mode_t is available in SA params and if

> the SA is configured in outbound direction and linked to the pktio

> then the configuration of dest_mode cannot be ODP_IPSEC_DEST_QUEUE.

> 

> >

> > -Petri

> >

> >

> >

On 23 March 2017 at 21:10, Peltonen, Janne (Nokia - FI/Espoo)
<janne.peltonen@nokia.com> wrote:
>> > This is part of odp_ipsec_sa_param_t, so lookup config is per SA.

>>

>> Does that mean that both SPI lookup and DEST ADDR lookup are mandatory?

>> Is there a reason lookup mode is per SA?

>

> The lookup mode is per SA to make it possible to have SAs for which

> not ODP but the application performs the lookup (e.g. currently for

> multicast IPsec SAs that do not have unique SPIs and require src

> address lookup too, or for any other SAs with weird lookup rules)

> and uses look-a-side IPsec ops with an explicit SA to do the

> IPsec transforms.

>

> Thus, I think at minimum the ODP_IPSEC_LOOKUP_DISABLED needs to

> be per-SA even if the SPI versus SPI+dstaddr selection would be

> global.


If I understand your requirement correctly then we could add a boolean
"enable_lookup" per SA which when disabled will
remove the SA from any lookup and we can configure the lookup mode as
a global configuration.

Regards,
Bala

>

>         Janne

>

>

>> -----Original Message-----

>> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of Bala Manoharan

>> Sent: Thursday, March 23, 2017 4:42 PM

>> To: Savolainen, Petri (Nokia - FI/Espoo) <petri.savolainen@nokia-bell-labs.com>

>> Cc: lng-odp-forward <lng-odp@lists.linaro.org>

>> Subject: Re: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline IPSEC support

>>

>> Regards,

>> Bala

>>

>>

>> On 23 March 2017 at 17:40, Savolainen, Petri (Nokia - FI/Espoo)

>> <petri.savolainen@nokia-bell-labs.com> wrote:

>> >> >

>> >> >  /**

>> >> > @@ -381,11 +547,29 @@ typedef enum odp_ipsec_lookup_mode_t {

>> >> >         ODP_IPSEC_LOOKUP_DISABLED = 0,

>> >> >

>> >> >         /** Inbound SA lookup is enabled. Used SPI values must be

>> >> unique. */

>> >> > -       ODP_IPSEC_LOOKUP_IN_UNIQUE_SA

>> >> > +       ODP_IPSEC_LOOKUP_IN_UNIQUE_SPI,

>> >> > +

>> >> > +       /** Inbound SA lookup is enabled. Lookup matches both SPI and

>> >> > +         * destination IP address. Used SPI values must be unique. */

>> >> > +       ODP_IPSEC_LOOKUP_IN_DSTADDR_UNIQUE_SPI

>> >> >

>> >> >  } odp_ipsec_lookup_mode_t;

>> >>

>> >> odp_ipsec_lookup_mode_t is not added either in odp_ipsec_config() or

>> >> in odp_ipsec_capability().

>> >> I believe this should be added in both these struct?

>> >

>> >

>> > This is part of odp_ipsec_sa_param_t, so lookup config is per SA.

>>

>> Does that mean that both SPI lookup and DEST ADDR lookup are mandatory?

>> Is there a reason lookup mode is per SA?

>>

>> >

>> >

>> >>

>> >> >

>> >> >

>> >> >  /**

>> >> > + * Result event destination

>> >> > + */

>> >> > +typedef enum odp_ipsec_dest_mode_t {

>> >> > +       /** Destination for IPSEC result events is a queue. */

>> >> > +       ODP_IPSEC_DEST_QUEUE = 0,

>> >> > +

>> >> > +       /** Destination for IPSEC result events is the classifier.

>> >> > +        *  IPSEC capability 'cls_inline' determines if inline

>> >> classification

>> >> > +        *  is supported. */

>> >> > +       ODP_IPSEC_DEST_CLS

>> >> > +

>> >> > +} odp_ipsec_dest_mode_t;

>> >>

>> >> Should'nt we add a dest_mode ODP_IPSEC_DEST_PKTIO for outbound inline

>> >> when the packet is sent out through interface directly.

>> >

>> > This selection is for result events. For output direction, queue are the only option

>> (for events). Queue vs inline pktout is selected by odp_ipsec_out_enq() vs

>> odp_ipsec_out_inline(). Selection of output pktio (or TM queue in the future) is

>> parameters to odp_ipsec_out_inline().

>>

>> Yes. But the odp_ipsec_dest_mode_t is available in SA params and if

>> the SA is configured in outbound direction and linked to the pktio

>> then the configuration of dest_mode cannot be ODP_IPSEC_DEST_QUEUE.

>>

>> >

>> > -Petri

>> >

>> >

>> >

> -----Original Message-----

> From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of

> Peltonen, Janne (Nokia - FI/Espoo)

> Sent: Thursday, March 23, 2017 5:25 PM

> To: Nikhil Agarwal <nikhil.agarwal@nxp.com>; lng-odp@lists.linaro.org

> Subject: Suspected SPAM - Re: [lng-odp] [API-NEXT PATCH v2 2/3] api:

> ipsec: add inline IPSEC support

> 

> Hi,

> 

> A few quick comments below. Petri will probably comment the other points.

> 

> 	Janne

> 

> > -----Original Message-----

> > From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of

> Nikhil Agarwal

> > Sent: Thursday, March 23, 2017 2:02 PM

> > To: Petri Savolainen <petri.savolainen@linaro.org>; lng-

> odp@lists.linaro.org

> > Subject: Re: [lng-odp] [API-NEXT PATCH v2 2/3] api: ipsec: add inline

> IPSEC support

> >

> > Few initial comments:

> >

> > - For Mixing inline and async processing must maintain same SA context

> in HW and

> > implementation. In that case, we should shift to async SA creation so

> that while creating

> > an SA, HW    interface can return the context to sw and Async processing

> can be done for

> > that context.


I think synchronous creation is simpler to use and it should not be a problem if creation takes a bit longer time in inline case. It's not a fast path function.


> > - Why we want to add different level of header retention, we can add

> retain header or

> > discard.



" Select up to which protocol layer (at least) outer headers are retained ... "

So, it's OK to retain all layers always.

Application just informs the minimum requirement. If implementation needs to do a copy (in SW) to retain headers, it would be waste to always require all headers (e.g. >60 bytes with IPv6+options) to be retained, when application is interested in only e.g. first 14 bytes.



> > - Why parsing is optional and layer wise. Parsing should be same as on

> the pktio

> > interface. Even in the lookaside APIs L3 parsing was mandatory.

> Otherwise application will

> > have to do the parsing on its own. Moreover, application will never know

> L3 protocol if

> > there are no parse results.



Fixed parsing requirement is replaced also for the lookaside API. Parsing is done for the decapsulated packet. The resulting packet after IPSEC has always an IP header, thus the L3 offset must be always set (== the start of the resulting packet). So, for LAYER_NONE application would not require any parsing - just the starting point (L3 offset).


> > - Support for ESP/AH and Tunnel/Transport mode should be exposed via

> capability.


These are part of the protocol, not special features - so I think those should not be optional.


> > - What about enabling HASH distribution post IPSEC for FAT tunnel use

> case.


Maybe in future. But if/when classifier is added with hashing capability, it may not be needed after that.


-Petri