[Challenge] large files smart compare (was: Re: Squeak for I/O and Memory Intensive tasks )

Bob Arning arning at charm.net
Tue Jan 29 09:28:03 UTC 2002


On Tue, 29 Jan 2002 09:39:54 +0200 Yoel Jacobsen <yoel at emet.co.il> wrote:
>The attached file is an almost empty LDAPObject class with fromStr: class method only. This method parse a single LDIF entry. Look the the class information for the commands to create a 10K lines LDIF file and to parse it.
>
>Parsing 10K lines took me about 12 minutes in which the image was working on this as a single task. Profiling shoes the time is mostly spent on adding to collections.

On Tue, 29 Jan 2002 01:12:08 -0700 Jon Hylands <jon at huv.com> wrote:
>I filed your code into a 3.1 image (3828), and it took 11.8 seconds to
>parse the 10,000 lines...

Yoel,

If you really mean 12 *minutes*, then something is slowing you down big-time. Your test case is going to consume over 10M of memory while running. Could you be short of real memory available to squeak? If you change your test to do

	MessageTally spyOn: [
		blocks _ lines findBetweenSubStrs: { crcr  }.
		objs _ blocks collect: [ :bl | LDAPObject fromStr: bl ].
	].

What do you get at the end of the report? How does it differ from this:

**Memory**
	old			+10,229,352 bytes
	young		-1,790,776 bytes
	used		+8,438,576 bytes
	free		-8,438,576 bytes

**GCs**
	full			0 totalling 0ms (0.0% uptime)
	incr		467 totalling 1,878ms (13.0% uptime), avg 4.0ms
	tenures		220 (avg 2 GCs/tenure)
	root table	0 overflows

Cheers,
Bob



More information about the Squeak-dev mailing list