On Sun, Oct 28, 2012 at 12:39:34AM -0400, David T. Lewis wrote:
> On Sat, Oct 27, 2012 at 11:41:15PM +0100, Frank Shearar wrote:
> > On 27 October 2012 23:37,  <commits at source.squeak.org> wrote:
> > > Frank Shearar uploaded a new version of KernelTests to project The Inbox:
> > > http://source.squeak.org/inbox/KernelTests-fbs.235.mcz
> > >
> > > ==================== Summary ====================
> > >
> > > Name: KernelTests-fbs.235
> > > Author: fbs
> > > Time: 27 October 2012, 11:37:53.223 pm
> > > UUID: ab0d84db-c1bc-40d0-bd90-786dcd6989b5
> > > Ancestors: KernelTests-ul.234
> > >
> > > Mark this test as an expected failure for the 4.4 release. Unmark it at the start of the 4.5 cycle!
> > >
> > > =============== Diff against KernelTests-ul.234 ===============
> > >
> > > Item was added:
> > > + ----- Method: LargeNegativeIntegerTest>>expectedFailures (in category 'as yet unclassified') -----
> > > + expectedFailures
> > > +       ^ #(testMinimumNegativeIntegerArithmetic).!
> > 
> > IIRC this test can only pass with suitable VM changes being made.
> > http://bugs.squeak.org/view.php?id=7705 mentions that the fix is in
> > VMMaker-dtl.286, so does this mean that with the interpreter VM this
> > test will pass, but with Cog (the VM we use in CI tests) the test will
> > fail?
> 
> Yes, that's right. It will fail with older interpreter VMs also.
>

Frank,

If you want to try running the test suite with an interpreter VM, I
installed a locally built VM on box3.squeak.org that can be used to
drive the tests. This is installed as /usr/local/bin/squeak, and can
be used for running both 32-bit and 64-bit Squeak images.

This VM is built from latest VMMaker and squeakvm.org sources using
the ~lewis/VMUnixBuild/VMUnixBuild.st script (attached), which now seems
to work fairly reliably. It is *not* suitable for general distribution,
so I would appreciate if we can all be careful to refer to it as a local
build, and not as "the standard VM" or any words to that effect.

Note, the latest standard interpreter VM (*) is not working on our
box3.squeak.org server due to a Linux glibc compatibility issue
(/lib/libc.so.6: version `GLIBC_2.15' not found), hence the need for
a local build.

(*) http://squeakvm.org/unix/release/Squeak-4.10.2.2614-linux_i386.tar.gz

Dave

-------------- next part --------------
"         --- VMUnixBuild.st ---
          24 October 2012  dtl

Generate and build a standard interpreter VM on Linux. Requires the
usual Linux development tools and libraries to be installed, as well
as the Subversion version control client.

This is a script file that may be supplied to Squeak at image startup.
It is intended to be run in a clean working directory. A large number
of packages will be installed, so start with a freshly downloaded
Squeak image.

This file may also be used in a Squeak workspace so that the expressions
below can be evaluated step by step to illustrate the build process.
Portions of this script will use CommandShell to perform system commands.
These steps may be done in a normal unix shell window if preferred.

Run the script like this:

   $ /usr/local/bin/squeak myImage.image VMUnixBuild.st

"

"Variables used in this script. You may skip this if you are evaluating
this script in a workspace, because a workspace declares its own variables
automatically."
| exitWhenDone fullScreen exitWith log project shell shellWindow vmmTool cwd
	configCommand configCommandImage64 startTime endTime tarball |

" --- configuration --- "

"If exitWhenDone is true, image will exit without saving when this
script is complete or if an error is encountered."

"EDIT BELOW. Set to true if the image should exit without saving when
this script is complete, or set false if script should be interactive."

exitWhenDone := false. "<== EDIT THIS"

"It is easier to watch the proceedings in full screen mode."
fullScreen := false. "If true, set to full screen mode"

"The configure command will specify various build time options. Define it
here so that it can be easily changed."

"With the following command, configure will build a VM using the
sources that VMMaker will generate in the src directory."
configCommand := '../platforms/unix/cmake/configure --src=../src'.

"If certain plugins are causing problems, they can be explicity removed
from the configuration. For example, if CameraPlugin and WeDoPlugin are
causing a problem, they can be excluded with the following configure command."
"configCommand :=
  '../platforms/unix/cmake/configure --src=../src --without-CameraPlugin --without-WeDoPlugin'."

"Build a VM for 64-bit images by adding the --image64 flag."
configCommandImage64 := configCommand, ' --image64'.

" --- subroutines --- "

"Log block. Write a message with time stamp to standard output."
log := [:msg | FileStream stdout
	nextPutAll: DateAndTime now printString, ': ';
	nextPutAll: msg;
	nextPut: Character lf;
	flush].


"Exit block. When running as a script, exit the image. If interactive,
open a notifier and let the user decide what to do."
exitWith := [:reason | exitWhenDone
	ifTrue: [log value: reason asString.
			log value: 'exiting image'.
				Smalltalk snapshot: false andQuit: true]
	ifFalse: [self notify: reason asString]].

" --- start of script --- "

startTime := DateAndTime now.

"If running with a display, show as much as possible so the command
shell window will be visible (see configuration variable above)."
fullScreen ifTrue: [Project current fullScreenOn].

log value: 'starting VM builder image'.

"Install OSProcess"
(Smalltalk hasClassNamed: #OSProcess) ifFalse: [
	log value: 'begin install OSProcess'.
	[MCMcmUpdater updateFromRepositories: #('http://www.squeaksource.com/OSProcess')]
		on: Error
		do: [exitWith value: 'failed loading OSProcess'].
	log value: 'install OSProcess complete'].

"AioEventHandler presents a warning dialog if the plugin is not
present, as may be the case if we are running a Cog VM. Initialize
it now so the dialog is not invoked if we are running headless."
Smalltalk at: #AioEventHandler ifPresent: [:aio | aio perform: #initializeAioPluginPresent].

"Install CommandShell"
(Smalltalk hasClassNamed: #CommandShell) ifFalse: [
	log value: 'begin install CommandShell'.
	[MCMcmUpdater updateFromRepositories: #('http://www.squeaksource.com/CommandShell')]
		on: Error
		do: [exitWith value: 'failed loading CommandShell'].
	log value: 'install CommandShell complete'].

"Open a shell. This will be used for later command invocation."
shell := Smalltalk at: #CommandShell ifPresent: [:cs | cs new].
shellWindow := shell open.

"Expand the shell window so we can see what is happening."
shellWindow dependents do: [:dep |
	(dep isKindOf: SystemWindow)
		ifTrue: [dep height: (dep height * 1.4) asInteger]].

"Perform the remainder of the script in a separate process. This permits
the shell window to be updated in the foreground UI process while shell
commands are being performed."

[ "Background process block starts here"
	| complete |
	complete := false.

	"Make sure that Subversion is installed before proceeding"
	shell if: 'svn --version'
		then: [log value: 'Subversion is installed on this system']
		else: [log value: 'Subversion is not installed on this system, ',
					'required for obtaining platform sources'.
				log value: 'exiting'.
				exitWith value: 'Subversion client not installed'].

	"Save previous ./src directory"
	log value: 'save ./src directory if it exists'.
	shell if: 'test -e src'
		then: [shell script: 'rm -rf src-old
			mv src src-old']
		else: ["nothing to save"].

	"Get the latest version of the platforms sources from the master repository."
	shell if: 'test -e platforms'
		then: [log value: 'update ./platforms from repository'.
				shell if: 'svn update platforms'
					then: [log value: 'platforms source update complete'.]
					else: [log value: 'platforms source update failed, exiting'.
							exitWith value: 'platforms source update failed']]
		else: [log value: 'check out ./platforms from repository'.
				shell if: 'svn co http://squeakvm.org/svn/squeak/trunk/platforms'
					then: [log value: 'platforms source checkout complete'.]
					else: [log value: 'platforms source checkout failed, exiting'.
							exitWith value: 'platforms source checkout failed']].

	"Install VMMaker trunk for interpreter VM building"
	(Smalltalk hasClassNamed: #VMMaker)
		ifTrue: [log value: 'begin VMMaker updateFromServer'.
				[(Smalltalk at: #VMMaker) updateFromServer]
					on: Error
					do: [exitWith value: 'failed VMMaker updateFromServer'].
				log value: 'VMMaker updateFromServer complete']
		ifFalse: [log value: 'begin install VMMaker (trunk)'.
				[MCMcmUpdater updateFromRepositories: #('http://source.squeak.org/VMMaker')]
					on: Error
					do: [exitWith value: 'failed loading VMMaker'].
				(Smalltalk at: #VMMaker) initializeBrowserSupport. "activate Slang browsing"
				log value: 'install VMMaker complete'].

	"Open a VMMaker tool initialized with the default Unix plugins.
	This should be done after loading the platforms sources, because the
	VMMaker will initialize only those plugins for which platform support
	exists."
	vmmTool := Smalltalk at: #VMMakerTool ifPresent: [:vmm | vmm forUnix].

	"Generate new sources"
	log value: 'generate all sources'.
	vmmTool vmMaker deleteEntireGeneratedTree. "make sure ./src is empty"
	vmmTool vmMaker generateEntire.
	log value: 'source generation complete'.

	"Build a VM from sources for standard 32-bit images."
	cwd := FileDirectory default pathName.
	log value: 'create build directory ./build'.
	shell command: 'rm -rf build'. "delete previous build directory"
	shell command: 'mkdir build'.
	shell command: 'cd build'.
	"The configure command is defined earlier in this script, see above."
	log value: 'cd to build, then run ', configCommand.
	shell if: configCommand
		then: [log value: 'configure completed successfully'.
			log value: 'begin make'.
			shell if: 'make'
				then: [log value: 'make completed with successful VM build'.
					log value: 'to install, cd to ./build and do "make install"']
				else: [log value: 'error in make']]
		else: [log value: 'problem in configure, make will not be run'].
	shell command: 'cd ', cwd. "back to home"

	"Repeat the procedure with --image64 to build the VM for 64-bit images. Note
	that /usr/local/bin/squeak script will attempt to locate the correct VM to
	use for any given image. Note also that you do not need to have a 64-bit platform
	in order to run a 64-bit image. It may not be fast, but it will work, at least
	until you run out of address space on your 32-bit platform."
	log value: 'create build directory ./build64'.
	shell command: 'rm -rf build64'. "delete previous build directory"
	shell command: 'mkdir build64'.
	shell command: 'cd build64'.
	log value: 'cd to build64, then run ', configCommandImage64.
	shell if: configCommandImage64
		then: [log value: 'configure completed successfully'.
			log value: 'begin make'.
			shell if: 'make'
				then: [log value: 'make completed with successful VM build'.
					log value: 'to install, cd to ./build and do "make install"'.
					complete := true]
				else: [log value: 'error in make']]
		else: [log value: 'problem in configure, make will not be run'].
	shell command: 'cd ', cwd. "back to home"

	"If the compile was successful, the platforms and generated sources are
	reasonably consistent. Save them as a tarball that can be compiled on other
	unix platforms. Save only the unix platforms sources."
	complete ifTrue: [ | now month day hour minute second ts pl v svnversion vmmversion vername command |
		now := DateAndTime now.
		month := now month asString.
		day := now dayOfMonth asString.
		hour := now hour24 asString.
		minute := now minute asString.
		second := now second asString.
		ts := now year asString,
			(month size < 2 ifTrue: ['0', month] ifFalse: [month]),
			(day size < 2 ifTrue: ['0', day] ifFalse: [day]),
			(hour size < 2 ifTrue: ['0', hour] ifFalse: [hour]),
			(minute size < 2 ifTrue: ['0', minute] ifFalse: [minute])",
			(second size < 2 ifTrue: ['0', second] ifFalse: [second])".
		vmmversion := VMMaker versionString.
		"find SVN version as in platforms/unix/cmake/configure script"
		pl := ProxyPipeline command: 'svn info platforms/unix/ChangeLog | fgrep Revision:'.
		v := pl upToEndOfFile.
		pl closePipes.
		svnversion := (v findTokens: ' ') second allButLast.
		vername := 'Squeak-vm-unix-', vmmversion, '-', svnversion, '-unofficial-src'.
		tarball := 'TARBALL/', vername, '-', ts, '.tar.gz'.

		shell if: 'test -d vername'
			then: [shell command: 'rm -rf ', vername]
			else: [].
		shell command: 'mkdir ', vername. "temp directory for sources to be tarballed"

		shell if: 'test -d TARBALL'
			then: []
			else: [shell command: 'mkdir TARBALL'].

		"Copy files of interest, omitting the Subversion control files and omitting
		the unit/src directory that contains generated sources from the most recent
		official VM build on squeakvm.org. CommandShell does not handle shell escaping
		well enough to do this properly with find and cpio, so we will resort to
		the following crude workaround."
		shell command: 'find src platforms/Cross platforms/unix -depth | cpio -pdm ', vername. 
		shell command: 'rm -rf ', vername, '/unix/src'.
		shell command: 'find ', vername, ' -depth -name .svn | xargs rm -rf '.

		"Save the copied files in a tarball to provide sources that may be configured
		and built on another system."
		shell if: 'tar czvf ', tarball, ' ', vername
			then: [log value: 'unix tarball saved as ', tarball]
			else: [log value: 'failed to create ', tarball].

		shell if: 'test -e ', vername
			then: [shell command: 'rm -rf ', vername]
			else: [].
	].

	log value: 'platform sources are in ./platforms'.
	log value: 'generated sources are in ./src'.
	log value: 'VM build is in ./build'.
	log value: 'to install the VM, cd to ./build and type make install'.
	log value: 'VM for 64-bit images is is in ./build'.
	log value: 'to install the VM for 64-bit images, cd to ./build64 and type make install'.
	log value: 'both versions of the VM can be installed on the same machine'.
	log value: 'the squeak script (normally /usr/local/bin/squeak) determines the VM to use'.

	"See below for some of the documentation included in the platforms/unix source true.
	Please also visit squeakvm.org for further information."

	"The configure script is a shell script that invokes CMake to set up a build configuration.
	The help output for the configure script explains how it is used, and what parameters
	may be passed to it for various custom configurations, or to adapt to platform constraints.
	Show the help for configure in an edit window."
	shell command: 'platforms/unix/cmake/configure -help | edit'.

	"More detailed information about the CMake build process is in README.CMake."
	shell command: 'edit platforms/unix/README.CMake'.

	"You will find lots of other documentation in platforms/unix/doc, and the other platforms
	directories may have similar documentation (e.g. platforms/win32/docs). If you would
	like to make contributions to the unix platforms code, it is a good idea to read the coding
	guidelines from the doc directory here."
	shell command: 'edit platforms/unix/doc/README.Contributing'.
	shell command: 'history # commands executed so far in this shell:'.

	endTime := DateAndTime now.
	log value: 'VM builder image complete after ', (endTime - startTime).

	shell if: 'test -f ', tarball
		then: [log value: 'unix tarball saved as ', tarball]
		else: [log value: 'source update incomplete, no source tarball created'].

	exitWith value: 'source generation and VM build complete'.

] fork.