> From: Mats Nygren <nygren at sics.se>
> 
> I have tried to delete FFI-references fore awhile. Hiding files in other
> directories, removing code from files including from generated files,
> the mess is still growing. Surely this is not how it should work. Adding
> a particular module shouldn't be able to mess up the whole machine
> generation process.

Agreed.  I mentioned some of these problems in the message announcing
2.8pre2.  It would help enormously if:

  - file names were consistent, i.e. FooPlugin contains the plugin, with all
    the EXPORT declarations and suchlike; then

  - sqFooPrims contains any generated primitives on which FooPlugin depends;

  - thinks like Squeak3D didn't have zillions of smaller b3dXYZ files
    (with no obvious relation to their "dependent") as dependencies.

This last one really bothers me.  I can see no good reason why
Squeak3D should be generated along with the interp.c and other support
files, and then be included in the sqNamedPrims.  It has worked
perfectly as a module for ages, has never needed to be listed in the
sqNamedPrims, and lived quite happily in its own dir where the
dependencies were obvious and easy to manage.

Unless this situation is sorted out soon I'm going to abandon the part
of configure that deduces dependencies automatically, and insist on a
file that contains explicit dependency information for the plugins.

This will be a real pain, because it would have to be edited whenever
a plugin were added/removed/renamed/whatever.  But the alternative (a
growing list of completely random rules for figuring out which files
belong in which modules) is a worse pain, and utterly unmanageable.

> For a unix machine I see two alternatives (there should be more):
> 
> 1) Control the build process from within Squeak and have a description
> in Squeak-syntax of the VM. (What modules should be included etc) This
> would need OSProcess and probably be unix-specific, in the beginning at
> least.

Simply a file containing lines of the form

	NameOfMainPlugin.c [AuxPrimsFile1.c [AuxPrimsFile2.c [...]]]

would be more than enough to solve the problem, and isn't specific to
any OS.  (W.r.t. OPP: there's absolutely no need to use an elephant
gun just to squash a bug.)

> 2) Have a simple way of describing parameters for the build, from
> outside of Squeak. Again this would probably be unix-specific in the
> beginning.

Arranging for the generated files to take note of a -DPLUGIN symbol to
compile themselves in "external plugin mode", for example, would be
perfect.  I could trivially rebuild the sqNamedPrims.h file in the
process, to reflect which plugins were "externalised".

> Manually changing generated files should never be a part of describing
> things the new VM.

Agreed!

FWIW, anyone who has regenerated the source files from inside the
image will also notice a Unix-specific FFI prims file that comes out
in the mix.  If this files isn't deleted then lots of other things
will blow up (and probably go "pop!" too).

> I find this very irritating at the moment, having decided to work with
> the syntactic part of VM-generation

Yes!

> I don't want learn m4, configure, etc at the moment. (In fact I
> would like to avoid those for ever if possible)

Exactly!  That's part of *my* job, not yours.  And for the time being
the only realistic solutions are:

  - reorganise the naming and mix of files that are generated automatically
    along with interp.c; or

  - give up on trying to figure out what belongs where (which was working
    perfectly right up the final Mac 2.8 when suddenly Squeak3D, SoundCodec
    and the FFI stuff [named all wrong] popped out of the translation
    process) and use a hand-edited plugin description file instead.

> Could somebody fix this please.

For building when libffi isn't there the fix is trivial.  I'll do it
today.

For the organisation, that has to be fixed by somebody who has the
right to meddle with the files generated by/stored in the image.
Which isn't me.  Andreas and John Mc are probably the most likely
candidates.

> The particular problem with FFI, since according to Bert (below)
> this isn't necesserary what everybody wants, why not make configure
> arrange for it to be left out, if it isn't found on the machine?

This is the intention, and this is how I thought it worked.

But note again: I had to remove an unwanted file from the generation
process, and add a #ifdef NO_FFI_SUPPORT (or something similar, I
think I "autoconfified" the name to HAVE_LIBFFI -- but it was missing
completely in the first place, so I don't feel guilty about that)
which was simply missing in one of the generated files.  If you've
regenerated the support files from within the image then zillions of
things will (and should) break -- since the generated files are wrong.

> If this is difficult to arrange then some things is
> severely wrong with the current organization of the build process.

It's not difficult to arrange, it just requires a little time and
motivation from someone in charge of the image side of things.

> Thanks for the help so far. I do wish to be able to work with this
> without first becoming an expert of every detail of the build process.

I know how you feel.  (But as a minor consolation: if you've learned
something this week, you never know when you'll be really glad to have
that knowledge.)

> > Since FFI is a major security risc I normally don't want it to be compiled
> > into the VM. I also set SQUEAK_SECURE.  Am I paranoid? ;-)

It's not really that much more of a risk than the current named prims,
which give access to every single system call on the machine from
within Squeak.  (It just makes passing arguments to them a little less
problematic.)

Ciao,

Ian