| Message ID | 20191218233928.696037-1-raj.khem@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | cmake: Displace VERBOSE setting | expand |
Hi Khem, On Wed, 2019-12-18 at 15:39 -0800, Khem Raj wrote: > Setting VERBOSE always, while is fine on one hand for debugging, its > coming at an expense of creating lots and lots of logs, e.g. qtwebkit > compile logs alone with VERBOSE is 163MB, there are many other large > packages which use cmake e.g. WPE, webkitgtk etc which are in same range > with out this option on, the logs reduce to 861K and also speeds up > build a notch > > If user needs to enable this logs for targetted debugging debugging that > could be added via > > EXTRA_OECMAKE_BUILD += "--verbose" > > in recipe > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > Cc: Ross Burton <ross.burton@intel.com> I don't agree with the reasoning to always disable this by default. - it's way too useful in general. I can't count the number of times it was enough to look at the log.do_compile to figure out something is wrong. Now you have to recompile everything with this merged, and something that took 5 minutes to debug becomes an arduous task. If the Webkit build is too verbose, you should make that specific recipe less verbose. E.g. you could always set EXTRA_OECMAKE_BUILD, and just clear it out in the webkit recipe. Otherwise, based on the same reasoning, you should disable all logs, not just cmake logs: kernel, make output in general, bitbake logs. They all take up space. Seriously, those logs are useful, keeping in mind that often logs are collected on build-machines, and having easy access is a huge time-saver. Cheers, Andre' > --- > meta/classes/cmake.bbclass | 9 +++++++-- > 1 file changed, 7 insertions(+), 2 deletions(-) > > diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass > index 8ccb1eefc7..8a44a0c216 100644 > --- a/meta/classes/cmake.bbclass > +++ b/meta/classes/cmake.bbclass > @@ -181,9 +181,14 @@ cmake_do_configure() { > -Wno-dev > } > > +# To enable VERBOSE option to emit all kind of logs for a given recipe add following in it > +# > +# EXTRA_OECMAKE_BUILD += "--verbose" > +# > +# Then run do_compile again > cmake_runcmake_build() { > - bbnote ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} > - eval ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} > + bbnote ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} > + eval ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} > } > > cmake_do_compile() { > -- > 2.24.1 > -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On 19/12/2019 08:52, André Draszik wrote: > Hi Khem, > > On Wed, 2019-12-18 at 15:39 -0800, Khem Raj wrote: >> Setting VERBOSE always, while is fine on one hand for debugging, its >> coming at an expense of creating lots and lots of logs, e.g. qtwebkit >> compile logs alone with VERBOSE is 163MB, there are many other large >> packages which use cmake e.g. WPE, webkitgtk etc which are in same range >> with out this option on, the logs reduce to 861K and also speeds up >> build a notch >> >> If user needs to enable this logs for targetted debugging debugging that >> could be added via >> >> EXTRA_OECMAKE_BUILD += "--verbose" >> >> in recipe >> >> Signed-off-by: Khem Raj <raj.khem@gmail.com> >> Cc: Ross Burton <ross.burton@intel.com> > > I don't agree with the reasoning to always disable this by default. - it's > way too useful in general. I can't count the number of times it was enough > to look at the log.do_compile to figure out something is wrong. > > Now you have to recompile everything with this merged, and something that > took 5 minutes to debug becomes an arduous task. > > If the Webkit build is too verbose, you should make that specific recipe > less verbose. E.g. you could always set > EXTRA_OECMAKE_BUILD, and just clear > it out in the webkit recipe. > > Otherwise, based on the same reasoning, you should disable all logs, not just > cmake logs: kernel, make output in general, bitbake logs. They all take up space. > > Seriously, those logs are useful, keeping in mind that often logs are > collected on build-machines, and having easy access is a huge time-saver. Personally I agree with André. We have verbose logs for autotools and meson, so why is cmake special. As this is a webkit-specific issue for you, the neater fix would be to override this in webkit. Stepping back though, why are 163MB log files a problem? Are you archiving every log maybe? If so then there's an argument to be made for a global "logs should be verbose" toggle, that defaults to on but you can set to off. That way the kernel, autotools, etc can also follow the same logic and *all* logs can be smaller. Ross
On Thu, 2019-12-19 at 08:52 +0000, André Draszik wrote: > Hi Khem, > > On Wed, 2019-12-18 at 15:39 -0800, Khem Raj wrote: > > Setting VERBOSE always, while is fine on one hand for debugging, > > its > > coming at an expense of creating lots and lots of logs, e.g. > > qtwebkit > > compile logs alone with VERBOSE is 163MB, there are many other > > large > > packages which use cmake e.g. WPE, webkitgtk etc which are in same > > range > > with out this option on, the logs reduce to 861K and also speeds up > > build a notch > > > > If user needs to enable this logs for targetted debugging debugging > > that > > could be added via > > > > EXTRA_OECMAKE_BUILD += "--verbose" > > > > in recipe > > > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > > Cc: Ross Burton <ross.burton@intel.com> > > I don't agree with the reasoning to always disable this by default. - > it's > way too useful in general. I can't count the number of times it was > enough > to look at the log.do_compile to figure out something is wrong. > What details do you look into when builds dont break ? and how often? perhaps that will help in understanding the case. > Now you have to recompile everything with this merged, and something > that > took 5 minutes to debug becomes an arduous task. > there are two aspects to it, one you described and another one is where users and infra is just building it. They are not component developers rather consumers. I think latter is more common by far. > If the Webkit build is too verbose, you should make that specific > recipe > less verbose. E.g. you could always set > EXTRA_OECMAKE_BUILD, and just clear > it out in the webkit recipe. > perhaps that is doable and after current patch it can be set globally too. However, for an OE user its a bad experience 1. its not default cmake option to enable verbosity, so when someone builds their stuff with OE, they don't get good experience, I consider that added impediment to use OE. 2. People who enable it know what they are doing. > Otherwise, based on the same reasoning, you should disable all logs, > not just > cmake logs: kernel, make output in general, bitbake logs. They all > take up space. > if they are large perhaps they should be too. 163M is a lot of logs, with slow disks it degrades the compile time, helps killing the disks quicker, all post processing it delayed, you pay high price for something you will occasionally use. > Seriously, those logs are useful, keeping in mind that often logs are > collected on build-machines, and having easy access is a huge time- > saver. > One case where people look into these logs are when something breaks and all error messages are logged without this option as well. > > Cheers, > Andre' > > > > > --- > > meta/classes/cmake.bbclass | 9 +++++++-- > > 1 file changed, 7 insertions(+), 2 deletions(-) > > > > diff --git a/meta/classes/cmake.bbclass > > b/meta/classes/cmake.bbclass > > index 8ccb1eefc7..8a44a0c216 100644 > > --- a/meta/classes/cmake.bbclass > > +++ b/meta/classes/cmake.bbclass > > @@ -181,9 +181,14 @@ cmake_do_configure() { > > -Wno-dev > > } > > > > +# To enable VERBOSE option to emit all kind of logs for a given > > recipe add following in it > > +# > > +# EXTRA_OECMAKE_BUILD += "--verbose" > > +# > > +# Then run do_compile again > > cmake_runcmake_build() { > > - bbnote ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build > > '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} > > - eval ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build > > '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} > > + bbnote ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' > > "$@" -- ${EXTRA_OECMAKE_BUILD} > > + eval ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" > > -- ${EXTRA_OECMAKE_BUILD} > > } > > > > cmake_do_compile() { > > -- > > 2.24.1 > >
On Thu, 2019-12-19 at 12:01 +0000, Ross Burton wrote: > On 19/12/2019 08:52, André Draszik wrote: > > Hi Khem, > > > > On Wed, 2019-12-18 at 15:39 -0800, Khem Raj wrote: > > > Setting VERBOSE always, while is fine on one hand for debugging, > > > its > > > coming at an expense of creating lots and lots of logs, e.g. > > > qtwebkit > > > compile logs alone with VERBOSE is 163MB, there are many other > > > large > > > packages which use cmake e.g. WPE, webkitgtk etc which are in > > > same range > > > with out this option on, the logs reduce to 861K and also speeds > > > up > > > build a notch > > > > > > If user needs to enable this logs for targetted debugging > > > debugging that > > > could be added via > > > > > > EXTRA_OECMAKE_BUILD += "--verbose" > > > > > > in recipe > > > > > > Signed-off-by: Khem Raj <raj.khem@gmail.com> > > > Cc: Ross Burton <ross.burton@intel.com> > > > > I don't agree with the reasoning to always disable this by default. > > - it's > > way too useful in general. I can't count the number of times it was > > enough > > to look at the log.do_compile to figure out something is wrong. > > > > Now you have to recompile everything with this merged, and > > something that > > took 5 minutes to debug becomes an arduous task. > > > > If the Webkit build is too verbose, you should make that specific > > recipe > > less verbose. E.g. you could always set > > EXTRA_OECMAKE_BUILD, and just clear > > it out in the webkit recipe. > > > > Otherwise, based on the same reasoning, you should disable all > > logs, not just > > cmake logs: kernel, make output in general, bitbake logs. They all > > take up space. > > > > Seriously, those logs are useful, keeping in mind that often logs > > are > > collected on build-machines, and having easy access is a huge time- > > saver. > > Personally I agree with André. We have verbose logs for autotools > and > meson, so why is cmake special. As this is a webkit-specific issue > for > you, the neater fix would be to override this in webkit. > perhaps there should be a global production mode and curb these logs too. Since now we are citing webkit as an example solving it just for webkit does'nt seem right option. > Stepping back though, why are 163MB log files a problem? Are you > archiving every log maybe? If so then there's an argument to be > made > for a global "logs should be verbose" toggle, that defaults to on > but > you can set to off. That way the kernel, autotools, etc can also > follow > the same logic and *all* logs can be smaller. I agree on this point. Although I wonder if 80% problems can be solved by fixing cmake verbosity then that not a bad start. After this patch one could infact enable it globally too by setting it in config metadata, so we are just toggling defaults. > > Ross >
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass index 8ccb1eefc7..8a44a0c216 100644 --- a/meta/classes/cmake.bbclass +++ b/meta/classes/cmake.bbclass @@ -181,9 +181,14 @@ cmake_do_configure() { -Wno-dev } +# To enable VERBOSE option to emit all kind of logs for a given recipe add following in it +# +# EXTRA_OECMAKE_BUILD += "--verbose" +# +# Then run do_compile again cmake_runcmake_build() { - bbnote ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} - eval ${DESTDIR:+DESTDIR=${DESTDIR} }VERBOSE=1 cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} + bbnote ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} + eval ${DESTDIR:+DESTDIR=${DESTDIR} } cmake --build '${B}' "$@" -- ${EXTRA_OECMAKE_BUILD} } cmake_do_compile() {
Setting VERBOSE always, while is fine on one hand for debugging, its coming at an expense of creating lots and lots of logs, e.g. qtwebkit compile logs alone with VERBOSE is 163MB, there are many other large packages which use cmake e.g. WPE, webkitgtk etc which are in same range with out this option on, the logs reduce to 861K and also speeds up build a notch If user needs to enable this logs for targetted debugging debugging that could be added via EXTRA_OECMAKE_BUILD += "--verbose" in recipe Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Ross Burton <ross.burton@intel.com> --- meta/classes/cmake.bbclass | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) -- 2.24.1 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core