[Vm-dev] libcruft

[Nicolas Cellier]

So once again, compiling libssh2 on win64 is broken...
This library is necessary for Pharo VM.
For now it only works "thanks" (?) to the .thirdparty-cache we have on our
build slaves.

This situation prevents from integrating further work.
https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/242

Compiling unixy stuff for windows (mingw) target is tough.
Esteban and I already wasted some hours on it.
But it seems that last tentative did not succeed
https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/57e68cef046c7309261d81d8ad1c45577562f16b

IMO, we could as well use the prebuilt cygwin libraries, rather than trying
to compile by ourselves. This would have some advantages:
- remove the necessity for build cache
- accelerate the build workers
- be up-to-date security-wise
- focus our scarce resource to more Smalltalk-related things
- leverage the work of other teams
with maybe this drawback:
- do not completely manage the exact version of every VM component

Managing every VM component also means managing the exact configuration of
the build workers (compiler version, libraries versions, etc...), which we
do not anyway.

In the meantime, if we want to fix the build, we have to understand why the
link phase fails.

CCLD libssh2.la
/usr/lib/gcc/x86_64-w64-mingw32/6.4.0/../../../../x86_64-w64-mingw32/bin/ld:
cannot find -link

We never told to link ink, so what is going on?
Here come autotools and his very brave libtool, or should I say libcruft.
If you want to teach yourself "how to not program", then do you a favor:

    cd build.win64x64/pharo.cog.spur
    mvm -f
    cd build/third-party/libssh2-1.7.0
    less m4/lib-link.m4

Scary! and it's just a tiny part of the overall cruft.
It's clear that the production of artifacts in such conditions is already
beyond the limits of human capabilities.
An idea could be to teach bots to do it for us, but if we really teach them
to produce such cruft, then I have no great hope that we'll live in a
better world ;)

Instead of this pathetic tentative of rule the universe imperative
programming, in such case we could enter some prolog-like rules and let a
bot do the inference for us, that may scale a bit longer...

If I go into build.win64x64/pharo.cog.spur/build/third-party/libssh2-1.7.0
and inquire a bit, I see this:

    grep -r '\-link' . | less

./aclocal.m4:m4_include([m4/lib-link.m4])
./compile:    linker_opts="-link$linker_opts"
./config.status:archive_cmds='$CC -o $lib $libobjs $compiler_flags
`func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames='
./configure:        sed -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
-EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols >
$output_objdir/$soname.exp;
./configure:        sed -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
-EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp;
./configure:    archive_cmds='$CC -o $lib $libobjs $compiler_flags
`func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link -dll~linknames='
./docs/Makefile:        $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \
./docs/Makefile.in:     $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \
./example/Makefile:     $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \
./example/Makefile.in:  $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \
./libtool:archive_cmds="\$CC -o \$lib \$libobjs \$compiler_flags
\\\`func_echo_all \\\"\$deplibs\\\" | \$SED 's/ -lc\$//'\\\` -link
-dll~linknames="
./libtool:      # -O*, -flto*, -fwhopr*, -fuse-linker-plugin GCC link-time
optimization
./libtool:      -O*|-flto*|-fwhopr*|-fuse-linker-plugin)
./ltmain.sh:      # -O*, -flto*, -fwhopr*, -fuse-linker-plugin GCC
link-time optimization
./ltmain.sh:      -O*|-flto*|-fwhopr*|-fuse-linker-plugin)
./m4/lib-link.m4:# lib-link.m4 serial 13 (gettext-0.16.2)
./m4/libtool.m4:            sed -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
-EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols >
$output_objdir/$soname.exp;
./m4/libtool.m4:            sed -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
-EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp;
./m4/libtool.m4:        _LT_TAGVAR(archive_cmds, $1)='$CC -o $lib $libobjs
$compiler_flags `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link
-dll~linknames='
./m4/libtool.m4:              $SED -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
-EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols >
$output_objdir/$soname.exp;
./m4/libtool.m4:              $SED -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
-EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp;
./m4/libtool.m4:*\ -fuse-linker-plugin*\ *) CFLAGS="$CFLAGS
-fno-use-linker-plugin" ;;
./Makefile:     $(top_srcdir)/m4/lib-ld.m4 $(top_srcdir)/m4/lib-link.m4 \
./Makefile:       ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
./Makefile.in:  $(top_srcdir)/m4/lib-ld.m4 $(top_srcdir)/m4/lib-link.m4 \
./Makefile.in:    ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \; -o \
./src/CMakeLists.txt:  # Fall back to over-linking dependencies
./src/Makefile: $(top_srcdir)/m4/lib-ld.m4 $(top_srcdir)/m4/lib-link.m4 \
./src/Makefile.in:      $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \
./tests/Makefile:       $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \
./tests/Makefile.in:    $(top_srcdir)/m4/lib-ld.m4
$(top_srcdir)/m4/lib-link.m4 \

Oh my, I don't even want to decipher that, it's obviously write-only.
All I understand is that the autoshit could decide to insert a -link by
itself and make our build fail at link phase in certain conditions...
It remains to see what we recently introduce to trigger these conditions.
How to debug this, I have no idea (insert prints here and there...).
Good luck.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180408/98470948/attachment.html>