[Vm-dev] Rant about generated files: should they stay in repository?

Fri Mar 11 23:36:04 UTC 2011

On Fri, 11 Mar 2011, John McIntosh wrote:

>
> The problem of having the generated files in or not in the repository
> clouds the real problem.
> In reflection what you really need is the ability for anyone to do
>
> squeakVMGeneratorBuilderWhatever generate: 'OSXCog" version: 5.8.2.12
>
> And what ever that process is will fetch generate and enable the
> client to actually have 5.8.2.12 of the OSX Cog VM compile and ready
> to go...
>
> In my opinion that is the real problem here that you need to:
>
> (a) determining the VM version you have (this is murky btw).
> (b) ensuring the toolset need to build the VM is there
> (c) ensuring any third party tools needed to build are there. (and
> ensure the versions are sane).
> (d) running the process and determining that the build of 5.8.2.12 is sane.
>
> This harkens back to the days of Envy where you could say build me
> version 5.8.2.12 of product X and the tool would *always* ensure what
> came out was 5.8.2.12
>
> Frankly "the client" here doesn't care what happens or how the files
> are stored or constructed. He *does* care that if he thinks he is
> building 5.8.2.12 then that is what he will get once the process
> completes.

Even if this can be done on Mac and maybe on Windows, it's impossible to 
do it on Un*x. When it works for n different unix systems, there will be 
at least another n where it doesn't work.

Levente

>
>
> On Fri, Mar 11, 2011 at 6:25 AM, stephane ducasse
> <stephane.ducasse at gmail.com> wrote:
>>
>> stefan apparently you did not follow the lecture of Eliot because it takes less time than that :)
>>
>>> Hi:
>>>
>>> Even so you don't like the generated code, you should provide tarballs for certain versions so that people like me who do not know how to generate the code can just grab some version of the VM and compile a debug version easily.
>>>
>>> I just run into a problem that prevents the standard interpreter from loading one of our RoarVM images.
>>> And just going to http://squeakvm.org/unix/ and grabbing the tarball, compiling it with debugging settings and hacking in some instrumentation was enough to give me a first clue what went wrong.
>>>
>>> If I would have had to learn how to generate the code first, that would have taken a day instead of an hour.
>>>
>>> Thanks
>>> Stefan
>>>
>>> On 11 Mar 2011, at 13:19, Esteban Lorenzano wrote:
>>>
>>>>
>>>> I wasn't aware of that discussion, but I'm still disagree with their conclusions.
>>>> I'm certain that sources must be versioned. But sources are: VMMaker + configuration + platform files... generated files are not sources and to keep them in source tree is prone to error.
>>>> That's my feeling about this... anyway, I'm also not sure about re-discuss this :P
>>>>
>>>> best,
>>>> Esteban
>>>>
>>>>
>>>> El 11/03/2011, a las 8:50a.m., Frank Shearar escribió:
>>>>
>>>>> We've had this discussion before, in November last year - "VM packaging for Cog transition" - starting here: http://lists.squeakfoundation.org/pipermail/vm-dev/2010-November/005797.html
>>>>>
>>>>> frank
>>>>>
>>>>> On 2011/03/11 11:34, Esteban Lorenzano wrote:
>>>>>>
>>>>>> Igor, I'm with you :)
>>>>>> let's get rid of generated sources from repository!
>>>>>>
>>>>>> best,
>>>>>> Esteban
>>>>>>
>>>>>> El 10/03/2011, a las 10:01p.m., Igor Stasenko escribió:
>>>>>>
>>>>>>>
>>>>>>> I am against it , because it is going to nowhere.
>>>>>>>
>>>>>>> So let me elaborate...
>>>>>>>
>>>>>>> In Cog, we have an image which generating an universal sources for VM
>>>>>>> on all platforms.
>>>>>>> These generated files then put under source control.
>>>>>>>
>>>>>>> Pros:
>>>>>>> - to build VM you can just checkout and build
>>>>>>>
>>>>>>> Cons:
>>>>>>> - err , wait.. which exactly VM you going to build? can you use same
>>>>>>> sources for different builds?
>>>>>>>
>>>>>>> Yes, I can use same generated sources for everything, but depending on
>>>>>>> configuration and/or target operating system i can receive completely
>>>>>>> different results.
>>>>>>>
>>>>>>> Because, for recreating VM as an artifact to reproduce same
>>>>>>> circumstances as they were (when some bug was triggered) a generated
>>>>>>> sources snapshot is not enough.
>>>>>>>
>>>>>>> Simple example:
>>>>>>> - VM compiled for Macs using -O3 optimization are stable.
>>>>>>> In contrary to linux, where using -O2 with certain files makes your
>>>>>>> VM crash few (milli)seconds after startup.
>>>>>>>
>>>>>>> Second ( i don't like repeating myself but still ;):
>>>>>>> - the only way to make sure that VMMaker loads into our latest and
>>>>>>> greatest image(s) and producing same output(s) is to have someone to
>>>>>>> continuously try load it,
>>>>>>> then use it to generate sources, then compile the VM and then run
>>>>>>> tests for that VM.
>>>>>>> We have a generous guys in our community, who can do that for us. This
>>>>>>> is a Mr Hudson, and Mr Jenkins that joined party lately.
>>>>>>>
>>>>>>> I'd like to stress that generated files are just an intermediate data,
>>>>>>> which nobody interested in.
>>>>>>>
>>>>>>> Because as input what is relevant should be only :
>>>>>>> - a VMMaker
>>>>>>> - a platform-specific sources/libs
>>>>>>> - a build configuration
>>>>>>>
>>>>>>> and as output we always have:
>>>>>>> - VM
>>>>>>>
>>>>>>> what happens in the middle is really unimportant as long as you can
>>>>>>> guarantee reproducibility of  the same output from same inputs.
>>>>>>>
>>>>>>> VM functioning should not depend on some random noise which comes from
>>>>>>> unrelated sources like
>>>>>>> - generated source files line ending convention
>>>>>>> - the date stamp when VM was built
>>>>>>> - anything which is not listed in the inputs list above
>>>>>>> and if it does, we should eliminate it.
>>>>>>>
>>>>>>> Because if it does, then you don't have a stable builds. And having a
>>>>>>> snapshot of sources which kinda works and kinda stable
>>>>>>> worth nothing, because at any moment in ongoing development you can
>>>>>>> meet same issue which will pop up out of nowhere.
>>>>>>> And it will pop out exactly because you don't care to always do things
>>>>>>> from scratch , starting from image and VMMaker.
>>>>>>> Because it is so convenient to just pull the generated sources from
>>>>>>> repo and you done.
>>>>>>> And those silly stupids which trying to use VMMaker are not of your
>>>>>>> concern.. because when they fail to build stable VM (and they will),
>>>>>>> you can always tell them that they are using wrong sources and because
>>>>>>> you can't reproduce it (things is always working well for you, in your
>>>>>>> warm corner :)
>>>>>>>
>>>>>>> So, i think that keeping a generated files works as some kind of
>>>>>>> defensive policy which doesn't allows you to address certain
>>>>>>> problems for real.
>>>>>>> And sure thing, because we are lazy :)
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best regards,
>>>>>>> Igor Stasenko AKA sig.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>> --
>>> Stefan Marr
>>> Software Languages Lab
>>> Vrije Universiteit Brussel
>>> Pleinlaan 2 / B-1050 Brussels / Belgium
>>> http://soft.vub.ac.be/~smarr
>>> Phone: +32 2 629 2974
>>> Fax:   +32 2 629 3525
>>>
>>
>>
>
>
>
> -- 
> ===========================================================================
> John M. McIntosh <johnmci at smalltalkconsulting.com>
> Corporate Smalltalk Consulting Ltd.  http://www.smalltalkconsulting.com
> ===========================================================================
>