[Release] [Ann] Bob is building

[Keith Hodges]

Dear All,

Bob is taking shape, he can now build images, package and upload the
results. I have not yet posted this to squeak-dev, because I havent got
time to handle a flood of enquireys just yet.

Would anyone be willing to see if they can write the code to generate
one click images?

cheers

Keith

====================
Public Bob Server

Bob is now running at bob.warwick.st where he has 160Gb of space
available for building in.
His work is visible via http://bob.warwick.st and
ftp://squeak@bob.warwick.st (password is the same as the swiki).

At present it is necessary to be able to see what Bob is doing because
Bob is not clever enough to spot or terminate builds that get stuck. (An
OSProcess waitForCommand:withTimeout: would be useful). Secondly it is
useful to debug as builds progress by enabling the stepping option.
Thirdly some builds are not YET entirely unattended (e.g. squeak-dev).

To make this possible there is a vnc server running which allows remote
monitoring of Bob and the images which it launches. VNC is running on
display 1 (port 5901) password is the same as the squeak swiki (for now).

Bob can monitor squeaksource repositories rss feed and keep up to date
with changes in Monticello repositories, currently:

1)    "Packages" - where the published packages, and dependencies are
maintained.
2)    "Bob" - where build tasks, and build definitions are maintained.
3)     He also monitors "Mantis" for changes every 15 minutes.

Every 3 Minutes, he runs all the build tasks.

Each of the build tasks checks to see if any of their input images have
changed. Input images are usually in the form of a zip file as uploaded
to the final destination server (ftp.squeak.org)

This design, allows multiple Bob instances to run collaboratively in a
distributed fashion. Each Bob selects which builds it wants to make, and
whether or not it is responsible for doing the final upload. Thus the
Bob instances co-orperate, using the "creationTime" on the zip files as
a build triggering signal, as one uploads its completed build zip,
another downloads that zip for its own tasks.

Installing Bob
==========
Bob will be a downloadable image, built by himself!

Bob - Repositories/Packages
====================

Rio/File-Kernel
Rio/File-Base - provides file handling
Logging/Logging - provides logging front end
Logging/SimpleLog - provides logging back end
Sake/Sake - provides tasks and dependencies between builds
Sake/Sake-Scheduler - periodic task execution
Packages/Packages-Library - all the current package dependency definitions
Sake/Sake-Bob - the top level bob tasks
Bob/Bob-Releases - user defined build tasks for official releases
Bob/Bob-SomeProduct - user defined bob build tasks for your own products
(public)
YourRepo/Bob-YourBuilds - user defined bob build tasks for your own
products (private - run bob yourself).

Bob - Server Configuration
===================

All local configuration is done in class BobConfig, so if you are
running your own Bob build/test server you can save and load all local
settings as a class file out of BobConfig.

examples:

BobConfig class >> #configBaseDir

    ^ '/bob' asDirectory

BobConfig class >> #configVm

    "the mac vm doesnt like to be opened via a symbolic link"
   
    ^ '/bob/vm/Squeak\ 3.8.20beta1U.app/Contents/MacOS/Squeak\ VM\ Opt'

etc etc etc....

Bob - Build Customization
===================

Your image builds will be defined as subclasses of BobBuildImage, and
can be packaged and managed in the squeaksource/Bob repository. If a Bob
server instance is aware of your package, then he may do your building
for you. He will watch the Bob repository rss feed for changes, and will
respond accordingly.

Defining a Build a) Metadata b) build script
=================================

Each build has a series of meta data methods, one per build number, e.g:

build0000

    info name: 'Squeak3.10.2-build'.
    info when: '2009-02-21T00:19:28.182+00:00'.
   
    info by: 'kph'.
    info image: 'ftp://ftp.squeak.org/3.10/Squeak3.10.2*basic.zip'.
   
    info stepping: true.
    info release: false.

    info upload:
'ssh://updates@squeakfoundation.org/var/www/files/3.11/3.10.2-build/'.
   
    info comment: 'testing'.       

    "The code that follows is the central part of the script that is
used to build the image"
""""""
    Installer install: 'Tasks'.
""""""
    "self halt."
    Squeak310MakeBuildCandidate taskBuild run.
   
The above method defines metadata which tells bob, what the build is
called, when to run it, where to obtain the starting image, whether to
put "self halts" in at convenient places for debugging, and where to
upload the results.

When To Build - info when:
=====================
code in #isBuildNeeded articulates rules which allow you to.

1) Build once at a specific time.
2) Build if a specific file (specified as a url with wildcards accepted)
has changed.
3) Build if the starting image zip file (specified as a url with
wildcards accepted) has changed.
4) Build periodically every x, minutes/days.
5) Build with a new explicit build number change
6) Do not build.

The rules are evaluated as follows:
a) If the output directory does not exist do not build (6) ***
b) If "info when" = false or #dont then do not build. (6)
c) If "info when" is a time that is not up yet then do not build. (1)
d) If "info when" is a file match whose file is not found then do not
build. (2)
e) If "info image" zip file match is not found then do not build. (3)
f) If the "info when" is a file match has changed since he last build,
then a new build is needed. (2)
g) If the "info image" zip  has changed since he last build, then a new
build is needed. (3)
h) If the build number has changed a new build is needed. (5)
i) If "info when" is a duration, build if the given time has passed
since the previous build. (4)
j) When "info when" is a time, if the previous build is the same or more
modern do not build (1)

If you run your own bob server he will attempt to build ALL of the
defined tasks EVERY 3 minutes or so.

So your primary way for indicating which of the builds you want is to
create the output directory (option a) marked *** above.

What To Build - #theScript
====================

theScript

    script reset.
   
     self
        scriptTranscriptLogToFileLoad;
        scriptTranscriptLogToFileStart;
        scriptAddLPF;
        scriptAddInfoScript;
        scriptTranscriptLogToFileStop;
        scriptSaveImageAndQuit: true


The script is assembled using pre-prepared chunks like those above. You
can define #theScript for your builds.
#scriptAddInfoScript, incorporates the script elements that are appended
to the metadata method.
The script that this generates is written out and fed as an input to the
chosen image.

e,g.

"self halt." HTTPSocket httpFileIn:
'installer.pbwiki.com/f/TranscriptToFile.cs'.
!"self halt." Preferences setPreference: #logTranscriptToFile toValue: true.
!"self halt." HTTPSocket httpFileIn: 'installer.pbwiki.com/f/LPF.st'.
!    Installer install: 'Tasks'.

!
    Squeak310MakeBuildCandidate taskBuild run.
   

!"self halt." Preferences setPreference: #logTranscriptToFile toValue:
false.
! | image resuming |
"self halt."
SystemVersion newVersion: 'Squeak3.10.2-build'.
SystemVersion current date: '2009-02-21T02:03:58.358+00:00' asDateAndTime.
image := SmalltalkImage current.
image cleanUpAllExcept: #(ChangeSet).
resuming := image saveAs: '090221-0203_Squeak3.10.2-build.image'.
resuming ifFalse: [ image snapshot: false andQuit: true ].

!

Note that when you write these script snippets they are expected to have
"self halt." in them at strategic points. If the "info stepping" flag is
set to true then these are uncommented.

What To Build - Build order #dependsOn:
==============================

The tasks are run in an order which suits all of the dependencies
explicitly stated.

self dependsOn: { BobBuild310Build build }

Manually Running the build:
=====================

[ BobBuild taskBuildAll run ] fork.

and if immediate uploading is turned off, #configUploadImmediately, a
manual upload can be performed later via.

BobBuild310Dev upload run.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.squeakfoundation.org/pipermail/release/attachments/20090222/5f9f26b3/attachment.htm