| Message ID | 20210702111825.491065-1-memxor@gmail.com |
|---|---|
| Headers | show |
| Series | Generic XDP improvements | expand |
On 02/07/2021 13.18, Kumar Kartikeya Dwivedi wrote: > This change implements CPUMAP redirect support for generic XDP programs. > The idea is to reuse the cpu map entry's queue that is used to push > native xdp frames for redirecting skb to a different CPU. This will > match native XDP behavior (in that RPS is invoked again for packet > reinjected into networking stack). > > To be able to determine whether the incoming skb is from the driver or > cpumap, we reuse skb->redirected bit that skips generic XDP processing > when it is set. To always make use of this, CONFIG_NET_REDIRECT guard on > it has been lifted and it is always available. > > From the redirect side, we add the skb to ptr_ring with its lowest bit > set to 1. This should be safe as skb is not 1-byte aligned. This allows > kthread to discern between xdp_frames and sk_buff. On consumption of the > ptr_ring item, the lowest bit is unset. > > In the end, the skb is simply added to the list that kthread is anyway > going to maintain for xdp_frames converted to skb, and then received > again by using netif_receive_skb_list. > > Bulking optimization for generic cpumap is left as an exercise for a > future patch for now. > > Since cpumap entry progs are now supported, also remove check in > generic_xdp_install for the cpumap. > > Reviewed-by: Toke Høiland-Jørgensen<toke@redhat.com> > Signed-off-by: Kumar Kartikeya Dwivedi<memxor@gmail.com> > --- > include/linux/bpf.h | 9 +++- > include/linux/skbuff.h | 10 +--- > kernel/bpf/cpumap.c | 116 ++++++++++++++++++++++++++++++++++------- > net/core/dev.c | 3 +- > net/core/filter.c | 6 ++- > 5 files changed, 114 insertions(+), 30 deletions(-) Acked-by: Jesper Dangaard Brouer <brouer@redhat.com>
On Fri, Jul 2, 2021 at 4:20 AM Kumar Kartikeya Dwivedi <memxor@gmail.com> wrote: > > This small series makes some improvements to generic XDP mode and brings it > closer to native XDP. Patch 1 splits out generic XDP processing into reusable > parts, patch 2 adds pointer friendly wrappers for bitops (not have to cast back > and forth the address of local pointer to unsigned long *), patch 3 implements > generic cpumap support (details in commit) and patch 4 allows devmap bpf prog > execution before generic_xdp_tx is called. > > Patch 5 just updates a couple of selftests to adapt to changes in behavior (in > that specifying devmap/cpumap prog fd in generic mode is now allowed). > > Changelog: > ---------- > v5 -> v6 > v5: https://lore.kernel.org/bpf/20210701002759.381983-1-memxor@gmail.com > * Put rcpu->prog check before RCU-bh section to avoid do_softirq (Jesper) Applied. Thanks
This small series makes some improvements to generic XDP mode and brings it closer to native XDP. Patch 1 splits out generic XDP processing into reusable parts, patch 2 adds pointer friendly wrappers for bitops (not have to cast back and forth the address of local pointer to unsigned long *), patch 3 implements generic cpumap support (details in commit) and patch 4 allows devmap bpf prog execution before generic_xdp_tx is called. Patch 5 just updates a couple of selftests to adapt to changes in behavior (in that specifying devmap/cpumap prog fd in generic mode is now allowed). Changelog: ---------- v5 -> v6 v5: https://lore.kernel.org/bpf/20210701002759.381983-1-memxor@gmail.com * Put rcpu->prog check before RCU-bh section to avoid do_softirq (Jesper) v4 -> v5 v4: https://lore.kernel.org/bpf/20210628114746.129669-1-memxor@gmail.com * Add comments and examples for new bitops macros (Alexei) v3 -> v4 v3: https://lore.kernel.org/bpf/20210622202835.1151230-1-memxor@gmail.com * Add detach now that attach of XDP program succeeds (Toke) * Clean up the test to use new ASSERT macros v2 -> v3 v2: https://lore.kernel.org/bpf/20210622195527.1110497-1-memxor@gmail.com * list_for_each_entry -> list_for_each_entry_safe (due to deletion of skb) v1 -> v2 v1: https://lore.kernel.org/bpf/20210620233200.855534-1-memxor@gmail.com * Move __ptr_{set,clear,test}_bit to bitops.h (Toke) Also changed argument order to match the bit op they wrap. * Remove map value size checking functions for cpumap/devmap (Toke) * Rework prog run for skb in cpu_map_kthread_run (Toke) * Set skb->dev to dst->dev after devmap prog has run * Don't set xdp rxq that will be overwritten in cpumap prog run Kumar Kartikeya Dwivedi (5): net: core: split out code to run generic XDP prog bitops: add non-atomic bitops for pointers bpf: cpumap: implement generic cpumap bpf: devmap: implement devmap prog execution for generic XDP bpf: tidy xdp attach selftests include/linux/bitops.h | 50 ++++++++ include/linux/bpf.h | 10 +- include/linux/netdevice.h | 2 + include/linux/skbuff.h | 10 +- include/linux/typecheck.h | 9 ++ kernel/bpf/cpumap.c | 116 +++++++++++++++--- kernel/bpf/devmap.c | 49 ++++++-- net/core/dev.c | 103 ++++++++-------- net/core/filter.c | 6 +- .../bpf/prog_tests/xdp_cpumap_attach.c | 43 +++---- .../bpf/prog_tests/xdp_devmap_attach.c | 39 +++--- 11 files changed, 298 insertions(+), 139 deletions(-)