[Vm-dev] libcruft

Mon Apr 9 06:13:14 UTC 2018

Le lun. 9 avr. 2018 à 00:27, Cyril Ferlicot D <cyril at ferlicot.me> a écrit :

>
> Le 08/04/2018 à 10:47, Nicolas Cellier a écrit :
> > So once again, compiling libssh2 on win64 is broken...
> > This library is necessary for Pharo VM.
> > For now it only works "thanks" (?) to the .thirdparty-cache we have on
> > our build slaves.
> >
> > This situation prevents from integrating further work.
> > https://github.com/OpenSmalltalk/opensmalltalk-vm/pull/242
> >
> > Compiling unixy stuff for windows (mingw) target is tough.
> > Esteban and I already wasted some hours on it.
> > But it seems that last tentative did not succeed
> >
> https://github.com/OpenSmalltalk/opensmalltalk-vm/commit/57e68cef046c7309261d81d8ad1c45577562f16b
> >
> > IMO, we could as well use the prebuilt cygwin libraries, rather than
> > trying to compile by ourselves. This would have some advantages:
> > - remove the necessity for build cache
> > - accelerate the build workers
> > - be up-to-date security-wise
> > - focus our scarce resource to more Smalltalk-related things
> > - leverage the work of other teams
> > with maybe this drawback:
> > - do not completely manage the exact version of every VM component
> >
>
> Hi Nicolas,
>
> You listed the pro to use a prebuilt cygwin libraries. Could you also
> list the cons for people like me who do not know them?
>
> I suppose that if it is not currently used there must be a reason.
>
> Thank you in advance.
>
> Hi Cyril,
The cons I thought about is just above: less control on exact version of
library used.
I think that this is the main reason, along with 2 others:
- symmetry with build on other platforms (Mac).
- historically, mingw was used for building, and since it is minimalist, it
does not have pre-built libraries, not even clang which was necessary for
building win64, and no support for legacy directx. That's why I had to
reintroduce the heavier cygwin (with mingw target, otherwise every user of
Pharo would have to install cygwin!).

A lot of incorrect configure fail to recognize a windows platform, or we
failed to guess the options to pass to configure... We have a few hacks and
workarounds in the Makefile.lib.extra but it is fragile, and the build
cache prevents us to detect regressions immediately.

> Managing every VM component also means managing the exact configuration
> > of the build workers (compiler version, libraries versions, etc...),
> > which we do not anyway.
> >
> > In the meantime, if we want to fix the build, we have to understand why
> > the link phase fails.
> >
> > CCLD libssh2.la <http://libssh2.la>
> >
> /usr/lib/gcc/x86_64-w64-mingw32/6.4.0/../../../../x86_64-w64-mingw32/bin/ld:
> > cannot find -link
> >
> > We never told to link ink, so what is going on?
> > Here come autotools and his very brave libtool, or should I say libcruft.
> > If you want to teach yourself "how to not program", then do you a favor:
> >
> >     cd build.win64x64/pharo.cog.spur
> >     mvm -f
> >     cd build/third-party/libssh2-1.7.0
> >     less m4/lib-link.m4
> >
> > Scary! and it's just a tiny part of the overall cruft.
> > It's clear that the production of artifacts in such conditions is
> > already beyond the limits of human capabilities.
> > An idea could be to teach bots to do it for us, but if we really teach
> > them to produce such cruft, then I have no great hope that we'll live in
> > a better world ;)
> >
> > Instead of this pathetic tentative of rule the universe imperative
> > programming, in such case we could enter some prolog-like rules and let
> > a bot do the inference for us, that may scale a bit longer...
> >
> > If I go into
> > build.win64x64/pharo.cog.spur/build/third-party/libssh2-1.7.0 and
> > inquire a bit, I see this:
> >
> >     grep -r '\-link' . | less
> >
> > ./aclocal.m4:m4_include([m4/lib-link.m4])
> > ./compile:    linker_opts="-link$linker_opts"
> > ./config.status:archive_cmds='$CC -o $lib $libobjs $compiler_flags
> > `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link
> -dll~linknames='
> > ./configure:        sed -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
> > -EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols >
> > $output_objdir/$soname.exp;
> > ./configure:        sed -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
> > -EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp;
> > ./configure:    archive_cmds='$CC -o $lib $libobjs $compiler_flags
> > `func_echo_all "$deplibs" | $SED '\''s/ -lc$//'\''` -link
> -dll~linknames='
> > ./docs/Makefile:        $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> > ./docs/Makefile.in:     $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> > ./example/Makefile:     $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> > ./example/Makefile.in:  $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> > ./libtool:archive_cmds="\$CC -o \$lib \$libobjs \$compiler_flags
> > \\\`func_echo_all \\\"\$deplibs\\\" | \$SED 's/ -lc\$//'\\\` -link
> > -dll~linknames="
> > ./libtool:      # -O*, -flto*, -fwhopr*, -fuse-linker-plugin GCC
> > link-time optimization
> > ./libtool:      -O*|-flto*|-fwhopr*|-fuse-linker-plugin)
> > ./ltmain.sh:      # -O*, -flto*, -fwhopr*, -fuse-linker-plugin GCC
> > link-time optimization
> > ./ltmain.sh:      -O*|-flto*|-fwhopr*|-fuse-linker-plugin)
> > ./m4/lib-link.m4:# lib-link.m4 serial 13 (gettext-0.16.2)
> > ./m4/libtool.m4:            sed -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
> > -EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols >
> > $output_objdir/$soname.exp;
> > ./m4/libtool.m4:            sed -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
> > -EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp;
> > ./m4/libtool.m4:        _LT_TAGVAR(archive_cmds, $1)='$CC -o $lib
> > $libobjs $compiler_flags `func_echo_all "$deplibs" | $SED '\''s/
> > -lc$//'\''` -link -dll~linknames='
> > ./m4/libtool.m4:              $SED -n -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
> > -EXPORT:\\\\\\\1/' -e '1\\\!p' < $export_symbols >
> > $output_objdir/$soname.exp;
> > ./m4/libtool.m4:              $SED -e 's/\\\\\\\(.*\\\\\\\)/-link\\\
> > -EXPORT:\\\\\\\1/' < $export_symbols > $output_objdir/$soname.exp;
> > ./m4/libtool.m4:*\ -fuse-linker-plugin*\ *) CFLAGS="$CFLAGS
> > -fno-use-linker-plugin" ;;
> > ./Makefile:     $(top_srcdir)/m4/lib-ld.m4 $(top_srcdir)/m4/lib-link.m4 \
> > ./Makefile:       ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \;
> -o \
> > ./Makefile.in:  $(top_srcdir)/m4/lib-ld.m4 $(top_srcdir)/m4/lib-link.m4 \
> > ./Makefile.in:    ! -type d ! -perm -444 -links 1 -exec chmod a+r {} \;
> -o \
> > ./src/CMakeLists.txt:  # Fall back to over-linking dependencies
> > ./src/Makefile: $(top_srcdir)/m4/lib-ld.m4 $(top_srcdir)/m4/lib-link.m4 \
> > ./src/Makefile.in:      $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> > ./tests/Makefile:       $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> > ./tests/Makefile.in:    $(top_srcdir)/m4/lib-ld.m4
> > $(top_srcdir)/m4/lib-link.m4 \
> >
> > Oh my, I don't even want to decipher that, it's obviously write-only.
> > All I understand is that the autoshit could decide to insert a -link by
> > itself and make our build fail at link phase in certain conditions...
> > It remains to see what we recently introduce to trigger these conditions.
> > How to debug this, I have no idea (insert prints here and there...).
> > Good luck.
> >
>
>
> --
> Cyril Ferlicot
> https://ferlicot.fr
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/vm-dev/attachments/20180409/a30bebc7/attachment-0001.html>