[v2,01/11] kbuild: userprogs: add nolibc support

Userprogs are built with the regular kernel compiler $CC.
A kernel compiler does not necessarily contain a libc which is required
for a normal userspace application.
However the kernel tree does contain a minimal libc implementation
"nolibc" which can be used to build userspace applications.

Introduce support to build userprogs against nolibc instead of the
default libc of the compiler, which may not exist.

Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>

---
This could probably be moved out of the generic kbuild makefiles.
I think the ergonimics would suffer and this functionality could be
used by other users of userprogs.

Also this does currently not support out-of-tree builds.
For that tools/include/nolibc/*.h and usr/include/*.h would need to be
installed into the build directory.
---
 Documentation/kbuild/makefiles.rst | 12 ++++++++++++
 scripts/Makefile.userprogs         | 16 +++++++++++++---
 2 files changed, 25 insertions(+), 3 deletions(-)

On Mon, May 26, 2025 at 04:40:17PM +0200, Thomas Weißschuh wrote:
> On Mon, May 26, 2025 at 04:19:53PM +0200, Nicolas Schier wrote:
> > On Mon, Apr 07, 2025 at 09:42:38AM +0200, Thomas Weißschuh wrote:
> > > Userprogs are built with the regular kernel compiler $CC.
> > > A kernel compiler does not necessarily contain a libc which is required
> > > for a normal userspace application.
> > > However the kernel tree does contain a minimal libc implementation
> > > "nolibc" which can be used to build userspace applications.
> > > 
> > > Introduce support to build userprogs against nolibc instead of the
> > > default libc of the compiler, which may not exist.
> > > 
> > > Signed-off-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de>
> > > 
> > > ---
> > > This could probably be moved out of the generic kbuild makefiles.
> > > I think the ergonimics would suffer and this functionality could be
> > > used by other users of userprogs.
> > > 
> > > Also this does currently not support out-of-tree builds.
> > 
> > (out-of-tree == external kmods;  out-of-source == build-dir != source-dir)
> > 
> > you probably meant out-of-source.
> 
> I *did* mean out-of-tree.
> 
> Out-of-source already works with the current patchset. It is the default setup of kunit.py.
> 
> > > For that tools/include/nolibc/*.h and usr/include/*.h would need to be
> > > installed into the build directory.
> > 
> > Out-of-source builds could be achieved by adding 'headers' as 
> > dependency, see below.
> > 
> > > ---
> > >  Documentation/kbuild/makefiles.rst | 12 ++++++++++++
> > >  scripts/Makefile.userprogs         | 16 +++++++++++++---
> > >  2 files changed, 25 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/Documentation/kbuild/makefiles.rst b/Documentation/kbuild/makefiles.rst
> > > index 3b9a8bc671e2e92126857059e985d6e5b2c43fd4..f905a6f77c965311c491cfd7ab3103185af7e82e 100644
> > > --- a/Documentation/kbuild/makefiles.rst
> > > +++ b/Documentation/kbuild/makefiles.rst
> > > @@ -970,6 +970,18 @@ When linking bpfilter_umh, it will be passed the extra option -static.
> > >  
> > >  From command line, :ref:`USERCFLAGS and USERLDFLAGS <userkbuildflags>` will also be used.
> > >  
> > > +Bulding userprogs against nolibc
> > 
> > Bulding -> Building
> 
> Ack.
> 
> > > +--------------------------------
> > > +
> > > +Not all kernel toolchains provide a libc.
> > > +Simple userprogs can be built against a very simple libc call "nolibc" provided
> > > +by the kernel source tree.
> > > +
> > > +Example::
> > > +
> > > +  # lib/kunit/Makefile
> > > +  uapi-preinit-nolibc := $(CONFIG_ARCH_HAS_NOLIBC)
> > > +
> > >  When userspace programs are actually built
> > >  ------------------------------------------
> > >  
> > > diff --git a/scripts/Makefile.userprogs b/scripts/Makefile.userprogs
> > > index f3a7e1ef3753b54303718fae97f4b3c9d4eac07c..a1447c02b948901631098b585f5cf4d3ea383a57 100644
> > > --- a/scripts/Makefile.userprogs
> > > +++ b/scripts/Makefile.userprogs
> > > @@ -16,10 +16,20 @@ user-csingle	:= $(addprefix $(obj)/, $(user-csingle))
> > >  user-cmulti	:= $(addprefix $(obj)/, $(user-cmulti))
> > >  user-cobjs	:= $(addprefix $(obj)/, $(user-cobjs))
> > >  
> > > +user-libgcc     := $(call try-run,$(CC) -Werror $(KBUILD_USERCFLAGS) -lgcc -x c -shared /dev/null -o "$$TMP",-lgcc)
> > > +
> > > +user_nolibc_ccflags := -nostdlib -nostdinc -static -fno-ident -fno-asynchronous-unwind-tables \
> > > +		      -ffreestanding -fno-stack-protector \
> > > +		      -isystem $(objtree)/usr/include -include $(srctree)/tools/include/nolibc/nolibc.h -isystem $(srctree)/tools/include/nolibc/
> > > +user_nolibc_ldflags := -nostdlib -nostdinc -static
> > > +user_nolibc_ldlibs  := $(user-libgcc)
> > > +
> > >  user_ccflags	= -Wp,-MMD,$(depfile) $(KBUILD_USERCFLAGS) $(userccflags) \
> > > -			$($(target-stem)-userccflags)
> > > -user_ldflags	= $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags)
> > > -user_ldlibs	= $(userldlibs) $($(target-stem)-userldlibs)
> > > +			$($(target-stem)-userccflags) $(if $($(target-stem)-nolibc),$(user_nolibc_ccflags))
> > > +user_ldflags	= $(KBUILD_USERLDFLAGS) $(userldflags) $($(target-stem)-userldflags) \
> > > +			$(if $($(target-stem)-nolibc),$(user_nolibc_ldflags))
> > > +user_ldlibs	= $(userldlibs) $($(target-stem)-userldlibs) \
> > > +			$(if $($(target-stem)-nolibc),$(user_nolibc_ldlibs))
> > >  
> > >  # Create an executable from a single .c file
> > >  quiet_cmd_user_cc_c = CC [U]  $@
> > 
> > Adding another hunk for scripts/Makefile.userprogs would allow to build
> > out-of-source:
> > 
> > @@ -39,5 +49,5 @@ $(call multi_depend, $(user-cmulti), , -objs)
> >  # Create .o file from a .c file
> >  quiet_cmd_user_cc_o_c = CC [U]  $@
> >        cmd_user_cc_o_c = $(CC) $(user_ccflags) -c -o $@ $<
> > -$(user-cobjs): $(obj)/%.o: $(src)/%.c FORCE
> > +$(user-cobjs): $(obj)/%.o: $(src)/%.c headers FORCE
> >         $(call if_changed_dep,user_cc_o_c)
> > 
> > But I am unsure if it is ok to add 'headers' as a build dependency for 
> > userprogs.  For me, it feels a bit odd, but I think it really makes 
> > sense here.
> 
> Currently this dependency is encoded in Kconfig.
> If CONFIG_HEADERS_INSTALL=y then the headers are installed in the 'prepare'
> phase and already available when building any userprog.
> To me this seems like the easier and nicer implementation.

I am sure, I had an out-of-source test build that failed due to missing 
header files -- but I can't reproduce it any more and yes, 
CONFIG_HEADERS_INSTALL should really be enough and better in several 
ways.

Sorry for the noice.

Kind regards,
Nicolas