[Vm-dev] Rant about generated files: should they stay in repository?

[Levente Uzonyi]

On Fri, 11 Mar 2011, Igor Stasenko wrote:

>
> I am against it , because it is going to nowhere.
>
> So let me elaborate...
>
> In Cog, we have an image which generating an universal sources for VM
> on all platforms.
> These generated files then put under source control.
>
> Pros:
> - to build VM you can just checkout and build
>
> Cons:
> - err , wait.. which exactly VM you going to build? can you use same
> sources for different builds?

It depends on the revision, doesn't it? If the sources are generated for 
every build (which is the case), then you can check out any version you 
want and you'll get matching platform files and generated sources.

>
> Yes, I can use same generated sources for everything, but depending on
> configuration and/or target operating system i can receive completely
> different results.
>
> Because, for recreating VM as an artifact to reproduce same
> circumstances as they were (when some bug was triggered) a generated
> sources snapshot is not enough.
>
> Simple example:
>  - VM compiled for Macs using -O3 optimization are stable.
> In contrary to linux, where using -O2 with certain files makes your
> VM crash few (milli)seconds after startup.

So, will generating the sources from VMMaker magically fix all bugs in the 
C compiler, libraries and OS?

>
> Second ( i don't like repeating myself but still ;):
> - the only way to make sure that VMMaker loads into our latest and
> greatest image(s) and producing same output(s) is to have someone to
> continuously try load it,
> then use it to generate sources, then compile the VM and then run
> tests for that VM.

This is image dependent. Will you add all images to your build server to 
make sure if VMMaker works in all of them?

> We have a generous guys in our community, who can do that for us. This
> is a Mr Hudson, and Mr Jenkins that joined party lately.
>
> I'd like to stress that generated files are just an intermediate data,
> which nobody interested in.
>
> Because as input what is relevant should be only :
> - a VMMaker
> - a platform-specific sources/libs
> - a build configuration
>
> and as output we always have:
>  - VM
>
> what happens in the middle is really unimportant as long as you can
> guarantee reproducibility of  the same output from same inputs.
>
> VM functioning should not depend on some random noise which comes from
> unrelated sources like
> - generated source files line ending convention
> - the date stamp when VM was built
> - anything which is not listed in the inputs list above
> and if it does, we should eliminate it.
>
> Because if it does, then you don't have a stable builds. And having a
> snapshot of sources which kinda works and kinda stable
> worth nothing, because at any moment in ongoing development you can
> meet same issue which will pop up out of nowhere.
> And it will pop out exactly because you don't care to always do things
> from scratch , starting from image and VMMaker.
> Because it is so convenient to just pull the generated sources from
> repo and you done.
> And those silly stupids which trying to use VMMaker are not of your
> concern.. because when they fail to build stable VM (and they will),
> you can always tell them that they are using wrong sources and because
> you can't reproduce it (things is always working well for you, in your
> warm corner :)
>
> So, i think that keeping a generated files works as some kind of
> defensive policy which doesn't allows you to address certain
> problems for real.
> And sure thing, because we are lazy :)

This is nonsense IMHO. You don't have to use the intermediate sources, 
you can generate the C sources from VMMaker if you want. And if you find a 
problem related to "line ending convention" or "the date stamp when VM was 
built", then you'll have a chance to find the cause of it by comparing 
your generated sources with the sources in svn.


Levente

>
>
> -- 
> Best regards,
> Igor Stasenko AKA sig.
>