Dear All,
Bob is taking shape, he can now build images, package and upload the results. I have not yet posted this to squeak-dev, because I havent got time to handle a flood of enquireys just yet.
Would anyone be willing to see if they can write the code to generate one click images?
cheers
Keith
==================== Public Bob Server
Bob is now running at bob.warwick.st where he has 160Gb of space available for building in. His work is visible via http://bob.warwick.st and ftp://squeak@bob.warwick.st (password is the same as the swiki).
At present it is necessary to be able to see what Bob is doing because Bob is not clever enough to spot or terminate builds that get stuck. (An OSProcess waitForCommand:withTimeout: would be useful). Secondly it is useful to debug as builds progress by enabling the stepping option. Thirdly some builds are not YET entirely unattended (e.g. squeak-dev).
To make this possible there is a vnc server running which allows remote monitoring of Bob and the images which it launches. VNC is running on display 1 (port 5901) password is the same as the squeak swiki (for now).
Bob can monitor squeaksource repositories rss feed and keep up to date with changes in Monticello repositories, currently:
1) "Packages" - where the published packages, and dependencies are maintained. 2) "Bob" - where build tasks, and build definitions are maintained. 3) He also monitors "Mantis" for changes every 15 minutes.
Every 3 Minutes, he runs all the build tasks.
Each of the build tasks checks to see if any of their input images have changed. Input images are usually in the form of a zip file as uploaded to the final destination server (ftp.squeak.org)
This design, allows multiple Bob instances to run collaboratively in a distributed fashion. Each Bob selects which builds it wants to make, and whether or not it is responsible for doing the final upload. Thus the Bob instances co-orperate, using the "creationTime" on the zip files as a build triggering signal, as one uploads its completed build zip, another downloads that zip for its own tasks.
Installing Bob ========== Bob will be a downloadable image, built by himself!
Bob - Repositories/Packages ====================
Rio/File-Kernel Rio/File-Base - provides file handling Logging/Logging - provides logging front end Logging/SimpleLog - provides logging back end Sake/Sake - provides tasks and dependencies between builds Sake/Sake-Scheduler - periodic task execution Packages/Packages-Library - all the current package dependency definitions Sake/Sake-Bob - the top level bob tasks Bob/Bob-Releases - user defined build tasks for official releases Bob/Bob-SomeProduct - user defined bob build tasks for your own products (public) YourRepo/Bob-YourBuilds - user defined bob build tasks for your own products (private - run bob yourself).
Bob - Server Configuration ===================
All local configuration is done in class BobConfig, so if you are running your own Bob build/test server you can save and load all local settings as a class file out of BobConfig.
examples:
BobConfig class >> #configBaseDir
^ '/bob' asDirectory
BobConfig class >> #configVm
"the mac vm doesnt like to be opened via a symbolic link"
^ '/bob/vm/Squeak\ 3.8.20beta1U.app/Contents/MacOS/Squeak\ VM\ Opt'
etc etc etc....
Bob - Build Customization ===================
Your image builds will be defined as subclasses of BobBuildImage, and can be packaged and managed in the squeaksource/Bob repository. If a Bob server instance is aware of your package, then he may do your building for you. He will watch the Bob repository rss feed for changes, and will respond accordingly.
Defining a Build a) Metadata b) build script =================================
Each build has a series of meta data methods, one per build number, e.g:
build0000
info name: 'Squeak3.10.2-build'. info when: '2009-02-21T00:19:28.182+00:00'.
info by: 'kph'. info image: 'ftp://ftp.squeak.org/3.10/Squeak3.10.2*basic.zip'.
info stepping: true. info release: false.
info upload: 'ssh://updates@squeakfoundation.org/var/www/files/3.11/3.10.2-build/'.
info comment: 'testing'.
"The code that follows is the central part of the script that is used to build the image" """""" Installer install: 'Tasks'. """""" "self halt." Squeak310MakeBuildCandidate taskBuild run.
The above method defines metadata which tells bob, what the build is called, when to run it, where to obtain the starting image, whether to put "self halts" in at convenient places for debugging, and where to upload the results.
When To Build - info when: ===================== code in #isBuildNeeded articulates rules which allow you to.
1) Build once at a specific time. 2) Build if a specific file (specified as a url with wildcards accepted) has changed. 3) Build if the starting image zip file (specified as a url with wildcards accepted) has changed. 4) Build periodically every x, minutes/days. 5) Build with a new explicit build number change 6) Do not build.
The rules are evaluated as follows: a) If the output directory does not exist do not build (6) *** b) If "info when" = false or #dont then do not build. (6) c) If "info when" is a time that is not up yet then do not build. (1) d) If "info when" is a file match whose file is not found then do not build. (2) e) If "info image" zip file match is not found then do not build. (3) f) If the "info when" is a file match has changed since he last build, then a new build is needed. (2) g) If the "info image" zip has changed since he last build, then a new build is needed. (3) h) If the build number has changed a new build is needed. (5) i) If "info when" is a duration, build if the given time has passed since the previous build. (4) j) When "info when" is a time, if the previous build is the same or more modern do not build (1)
If you run your own bob server he will attempt to build ALL of the defined tasks EVERY 3 minutes or so.
So your primary way for indicating which of the builds you want is to create the output directory (option a) marked *** above.
What To Build - #theScript ====================
theScript
script reset.
self scriptTranscriptLogToFileLoad; scriptTranscriptLogToFileStart; scriptAddLPF; scriptAddInfoScript; scriptTranscriptLogToFileStop; scriptSaveImageAndQuit: true
The script is assembled using pre-prepared chunks like those above. You can define #theScript for your builds. #scriptAddInfoScript, incorporates the script elements that are appended to the metadata method. The script that this generates is written out and fed as an input to the chosen image.
e,g.
"self halt." HTTPSocket httpFileIn: 'installer.pbwiki.com/f/TranscriptToFile.cs'. !"self halt." Preferences setPreference: #logTranscriptToFile toValue: true. !"self halt." HTTPSocket httpFileIn: 'installer.pbwiki.com/f/LPF.st'. ! Installer install: 'Tasks'.
! Squeak310MakeBuildCandidate taskBuild run.
!"self halt." Preferences setPreference: #logTranscriptToFile toValue: false. ! | image resuming | "self halt." SystemVersion newVersion: 'Squeak3.10.2-build'. SystemVersion current date: '2009-02-21T02:03:58.358+00:00' asDateAndTime. image := SmalltalkImage current. image cleanUpAllExcept: #(ChangeSet). resuming := image saveAs: '090221-0203_Squeak3.10.2-build.image'. resuming ifFalse: [ image snapshot: false andQuit: true ].
!
Note that when you write these script snippets they are expected to have "self halt." in them at strategic points. If the "info stepping" flag is set to true then these are uncommented.
What To Build - Build order #dependsOn: ==============================
The tasks are run in an order which suits all of the dependencies explicitly stated.
self dependsOn: { BobBuild310Build build }
Manually Running the build: =====================
[ BobBuild taskBuildAll run ] fork.
and if immediate uploading is turned off, #configUploadImmediately, a manual upload can be performed later via.
BobBuild310Dev upload run.