[bpf-next,v4,0/6] bpf: BTF support for ksyms

v3 -> v4:
 - Rebasing
 - Cast bpf_[per|this]_cpu_ptr's parameter to void __percpu * before
   passing into per_cpu_ptr.

v2 -> v3:
 - Rename functions and variables in verifier for better readability.
 - Stick to logging message convention in libbpf.
 - Move bpf_per_cpu_ptr and bpf_this_cpu_ptr from trace-specific
   helper set to base helper set.
 - More specific test in ksyms_btf.
 - Fix return type cast in bpf_*_cpu_ptr.
 - Fix btf leak in ksyms_btf selftest.
 - Fix return error code for kallsyms_find().

v1 -> v2:
 - Move check_pseudo_btf_id from check_ld_imm() to
   replace_map_fd_with_map_ptr() and rename the latter.
 - Add bpf_this_cpu_ptr().
 - Use bpf_core_types_are_compat() in libbpf.c for checking type
   compatibility.
 - Rewrite typed ksym extern type in BTF with int to save space.
 - Minor revision of bpf_per_cpu_ptr()'s comments.
 - Avoid using long in tests that use skeleton.
 - Refactored test_ksyms.c by moving kallsyms_find() to trace_helpers.c
 - Fold the patches that sync include/linux/uapi and
   tools/include/linux/uapi.

rfc -> v1:
 - Encode VAR's btf_id for PSEUDO_BTF_ID.
 - More checks in verifier. Checking the btf_id passed as
   PSEUDO_BTF_ID is valid VAR, its name and type.
 - Checks in libbpf on type compatibility of ksyms.
 - Add bpf_per_cpu_ptr() to access kernel percpu vars. Introduced
   new ARG and RET types for this helper.

This patch series extends the previously added __ksym externs with
btf support.

Right now the __ksym externs are treated as pure 64-bit scalar value.
Libbpf replaces ld_imm64 insn of __ksym by its kernel address at load
time. This patch series extend those externs with their btf info. Note
that btf support for __ksym must come with the kernel btf that has
VARs encoded to work properly. The corresponding chagnes in pahole
is available at [1] (with a fix at [2] for gcc 4.9+).

The first 3 patches in this series add support for general kernel
global variables, which include verifier checking (01/06), libpf
support (02/06) and selftests for getting typed ksym extern's kernel
address (03/06).

The next 3 patches extends that capability further by introducing
helpers bpf_per_cpu_ptr() and bpf_this_cpu_ptr(), which allows accessing
kernel percpu variables correctly (04/06 and 05/06).

The tests of this feature were performed against pahole that is extended
with [1] and [2]. For kernel BTF that does not have VARs encoded, the
selftests will be skipped.

[1] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/commit/?id=f3d9054ba8ff1df0fc44e507e3a01c0964cabd42
[2] https://www.spinics.net/lists/dwarves/msg00451.html

Hao Luo (6):
  bpf: Introduce pseudo_btf_id
  bpf/libbpf: BTF support for typed ksyms
  selftests/bpf: ksyms_btf to test typed ksyms
  bpf: Introduce bpf_per_cpu_ptr()
  bpf: Introducte bpf_this_cpu_ptr()
  bpf/selftests: Test for bpf_per_cpu_ptr() and bpf_this_cpu_ptr()

 include/linux/bpf.h                           |   6 +
 include/linux/bpf_verifier.h                  |   7 +
 include/linux/btf.h                           |  26 +++
 include/uapi/linux/bpf.h                      |  67 +++++-
 kernel/bpf/btf.c                              |  25 ---
 kernel/bpf/helpers.c                          |  32 +++
 kernel/bpf/verifier.c                         | 190 ++++++++++++++++--
 kernel/trace/bpf_trace.c                      |   4 +
 tools/include/uapi/linux/bpf.h                |  67 +++++-
 tools/lib/bpf/libbpf.c                        | 112 +++++++++--
 .../testing/selftests/bpf/prog_tests/ksyms.c  |  38 ++--
 .../selftests/bpf/prog_tests/ksyms_btf.c      |  88 ++++++++
 .../selftests/bpf/progs/test_ksyms_btf.c      |  55 +++++
 tools/testing/selftests/bpf/trace_helpers.c   |  27 +++
 tools/testing/selftests/bpf/trace_helpers.h   |   4 +
 15 files changed, 653 insertions(+), 95 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/ksyms_btf.c
 create mode 100644 tools/testing/selftests/bpf/progs/test_ksyms_btf.c

Ah, this is the bug in pahole described in
https://lkml.org/lkml/2020/8/20/1862. I proposed a fix [1] but it
hasn't reached pahole's master branch. Let me ask Arnaldo to see if he
is OK merging it.

[1] https://www.spinics.net/lists/dwarves/msg00451.html

On Tue, Sep 29, 2020 at 9:36 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Tue, Sep 29, 2020 at 4:50 PM Hao Luo <haoluo@google.com> wrote:
> >
> > v3 -> v4:
> >  - Rebasing
> >  - Cast bpf_[per|this]_cpu_ptr's parameter to void __percpu * before
> >    passing into per_cpu_ptr.
>
> Looks good, but doesn't work:
> ./test_progs -t ksyms_btf
> test_ksyms_btf:PASS:kallsyms_fopen 0 nsec
> test_ksyms_btf:PASS:ksym_find 0 nsec
> test_ksyms_btf:PASS:kallsyms_fopen 0 nsec
> test_ksyms_btf:PASS:ksym_find 0 nsec
> test_ksyms_btf:PASS:btf_exists 0 nsec
> libbpf: extern (ksym) 'bpf_prog_active': incompatible types, expected
> [4] int int, but kernel has [18729] var bpf_user_rnd_state
> libbpf: failed to load object 'test_ksyms_btf'
> libbpf: failed to load BPF skeleton 'test_ksyms_btf': -22
> test_ksyms_btf:FAIL:skel_open failed to open and load skeleton
> #43 ksyms_btf:FAIL
>
> I have the latest pahole from master. Any ideas?

On Tue, Sep 29, 2020 at 11:48 PM Hao Luo <haoluo@google.com> wrote:
>
> Ah, this is the bug in pahole described in
> https://lkml.org/lkml/2020/8/20/1862. I proposed a fix [1] but it
> hasn't reached pahole's master branch. Let me ask Arnaldo to see if he
> is OK merging it.
>
> [1] https://www.spinics.net/lists/dwarves/msg00451.html
>
> On Tue, Sep 29, 2020 at 9:36 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Tue, Sep 29, 2020 at 4:50 PM Hao Luo <haoluo@google.com> wrote:
> > >
> > > v3 -> v4:
> > >  - Rebasing
> > >  - Cast bpf_[per|this]_cpu_ptr's parameter to void __percpu * before
> > >    passing into per_cpu_ptr.

I've rebased it myself and applied. Thanks Hao.

Thanks, Alexei and Andrii and other reviewers for the comments. It's a
pleasure to work with you and contribute to bpf.

Hao

On Fri, Oct 2, 2020 at 3:16 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>

> On Tue, Sep 29, 2020 at 11:48 PM Hao Luo <haoluo@google.com> wrote:

> >

> > Ah, this is the bug in pahole described in

> > https://lkml.org/lkml/2020/8/20/1862. I proposed a fix [1] but it

> > hasn't reached pahole's master branch. Let me ask Arnaldo to see if he

> > is OK merging it.

> >

> > [1] https://www.spinics.net/lists/dwarves/msg00451.html

> >

> > On Tue, Sep 29, 2020 at 9:36 PM Alexei Starovoitov

> > <alexei.starovoitov@gmail.com> wrote:

> > >

> > > On Tue, Sep 29, 2020 at 4:50 PM Hao Luo <haoluo@google.com> wrote:

> > > >

> > > > v3 -> v4:

> > > >  - Rebasing

> > > >  - Cast bpf_[per|this]_cpu_ptr's parameter to void __percpu * before

> > > >    passing into per_cpu_ptr.

>

> I've rebased it myself and applied. Thanks Hao.

On Tue, Sep 29, 2020 at 4:50 PM Hao Luo <haoluo@google.com> wrote:
>
> -       ret = replace_map_fd_with_map_ptr(env);
> -       if (ret < 0)
> -               goto skip_full_check;
> -
>         if (bpf_prog_is_dev_bound(env->prog->aux)) {
>                 ret = bpf_prog_offload_verifier_prep(env->prog);
>                 if (ret)
> @@ -11662,6 +11757,10 @@ int bpf_check(struct bpf_prog **prog, union bpf_attr *attr,
>         if (ret)
>                 goto skip_full_check;
>
> +       ret = resolve_pseudo_ldimm64(env);
> +       if (ret < 0)
> +               goto skip_full_check;
> +

Hao,

this change broke several tests in test_verifier:
#21/u empty prog FAIL
Unexpected error message!
    EXP: unknown opcode 00
    RES: last insn is not an exit or jmp

#656/u test5 ld_imm64 FAIL
Unexpected error message!
    EXP: invalid bpf_ld_imm64 insn
    RES: last insn is not an exit or jmp

#656/p test5 ld_imm64 FAIL
Unexpected error message!
    EXP: invalid bpf_ld_imm64 insn
    RES: last insn is not an exit or jmp

Please send a fix.
Thanks