[bpf-next] selftests/bpf: Cross-compile bpftool

From: Björn Töpel <bjorn@rivosinc.com>

When the BPF selftests are cross-compiled, only the a host version of
bpftool is built. This version of bpftool is used to generate various
intermediates, e.g., skeletons.

The test runners are also using bpftool. The Makefile will symlink
bpftool from the selftest/bpf root, where the test runners will look
for the tool:

  | ...
  | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
  |    $(OUTPUT)/$(if $2,$2/)bpftool

There are two issues for cross-compilation builds:

 1. There is no native (cross-compilation target) build of bpftool
 2. The bootstrap variant of bpftool is never cross-compiled (by
    design)

Make sure that a native/cross-compiled version of bpftool is built,
and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
version.

Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
---
 tools/testing/selftests/bpf/Makefile | 28 +++++++++++++++++++++++++---
 1 file changed, 25 insertions(+), 3 deletions(-)


base-commit: 06744f24696e1e7598412c3df61a538b57ebec22

2023-02-10 09:43 UTC+0100 ~ Björn Töpel <bjorn@kernel.org>
> From: Björn Töpel <bjorn@rivosinc.com>
> 
> When the BPF selftests are cross-compiled, only the a host version of
> bpftool is built. This version of bpftool is used to generate various
> intermediates, e.g., skeletons.
> 
> The test runners are also using bpftool. The Makefile will symlink
> bpftool from the selftest/bpf root, where the test runners will look
> for the tool:
> 
>   | ...
>   | $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
>   |    $(OUTPUT)/$(if $2,$2/)bpftool
> 
> There are two issues for cross-compilation builds:
> 
>  1. There is no native (cross-compilation target) build of bpftool
>  2. The bootstrap variant of bpftool is never cross-compiled (by
>     design)
> 
> Make sure that a native/cross-compiled version of bpftool is built,
> and if CROSS_COMPILE is set, symlink to the native/non-bootstrap
> version.
> 
> Signed-off-by: Björn Töpel <bjorn@rivosinc.com>
> ---
>  tools/testing/selftests/bpf/Makefile | 28 +++++++++++++++++++++++++---
>  1 file changed, 25 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index b2eb3201b85a..b706750f71e2 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile
> @@ -157,8 +157,9 @@ $(notdir $(TEST_GEN_PROGS)						\
>  	 $(TEST_CUSTOM_PROGS)): %: $(OUTPUT)/% ;
>  
>  # sort removes libbpf duplicates when not cross-building
> -MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	       \
> -	       $(HOST_BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/resolve_btfids      \
> +MAKE_DIRS := $(sort $(BUILD_DIR)/libbpf $(HOST_BUILD_DIR)/libbpf	\
> +	       $(BUILD_DIR)/bpftool $(HOST_BUILD_DIR)/bpftool		\
> +	       $(HOST_BUILD_DIR)/resolve_btfids				\
>  	       $(RUNQSLOWER_OUTPUT) $(INCLUDE_DIR))
>  $(MAKE_DIRS):
>  	$(call msg,MKDIR,,$@)
> @@ -208,6 +209,14 @@ $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(wildcard bpf_testmod/Makefile bpf_tes
>  	$(Q)cp bpf_testmod/bpf_testmod.ko $@
>  
>  DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
> +ifneq ($(CROSS_COMPILE),)
> +CROSS_BPFTOOL := $(SCRATCH_DIR)/sbin/bpftool
> +TRUNNER_BPFTOOL := $(CROSS_BPFTOOL)
> +USE_BOOTSTRAP := ""
> +else
> +TRUNNER_BPFTOOL := $(DEFAULT_BPFTOOL)
> +USE_BOOTSTRAP := "bootstrap"
> +endif
>  
>  $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT)
>  	$(Q)$(MAKE) $(submake_extras) -C $(TOOLSDIR)/bpf/runqslower	       \
> @@ -255,6 +264,18 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)    \
>  		    LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/			       \
>  		    prefix= DESTDIR=$(HOST_SCRATCH_DIR)/ install-bin
>  
> +ifneq ($(CROSS_COMPILE),)
> +$(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile)	\
> +		    $(BPFOBJ) | $(BUILD_DIR)/bpftool
> +	$(Q)$(MAKE) $(submake_extras)  -C $(BPFTOOLDIR)				\
> +		    ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE)			\
> +		    EXTRA_CFLAGS='-g -O0'					\
> +		    OUTPUT=$(BUILD_DIR)/bpftool/				\
> +		    LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/				\
> +		    LIBBPF_DESTDIR=$(SCRATCH_DIR)/				\
> +		    prefix= DESTDIR=$(SCRATCH_DIR)/ install-bin
> +endif
> +
>  all: docs
>  
>  docs:
> @@ -518,11 +539,12 @@ endif
>  $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)			\
>  			     $(TRUNNER_EXTRA_OBJS) $$(BPFOBJ)		\
>  			     $(RESOLVE_BTFIDS)				\
> +			     $(TRUNNER_BPFTOOL)				\
>  			     | $(TRUNNER_BINARY)-extras
>  	$$(call msg,BINARY,,$$@)
>  	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) -o $$@
>  	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
> -	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/bootstrap/bpftool \
> +	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)/bpftool \

Nit: You'll have a double slash in this path when USE_BOOSTRAP is empty
(.../tools/build/bpftool//bpftool), but it probably doesn't matter much.

>  		   $(OUTPUT)/$(if $2,$2/)bpftool
>  
>  endef
> 
> base-commit: 06744f24696e1e7598412c3df61a538b57ebec22

The changes look good to me, thanks!

Acked-by: Quentin Monnet <quentin@isovalent.com>

Jean-Philippe, I know you do some cross-compiling with bpftool, how does
this look from your side?

Hi Quentin,

On Mon, Feb 13, 2023 at 08:53:31PM +0000, Quentin Monnet wrote:
> Jean-Philippe, I know you do some cross-compiling with bpftool, how does
> this look from your side?

I don't have enough time for BPF at the moment unfortunately. Zachary is
looking at cross-compiling the selftests for arm64

Thanks,
Jean