counting tokens in a string

Jason Rogers jacaetevha at fast-mail.org
Mon May 6 22:39:17 UTC 2002


Many thanks.  I knew that Smalltalk should have a better way, it's just
that I don't know the class libraries well enough to know where to
look.  In fact, occurrences: was what I really wanted.  I should have
followed my gut and just typed it. :)

Funny though, there must be something wrong with my image/VM.  Many of
the primitive calls I indirectly call either lie to me, hang, or take
inordinate amounts of time.  For instance, StandardFileStream>>atEnd
never seems to tell me it's at the end.  However, doing '(stream
position < stream size) whileTrue:[...]' does work. 
StandardFileStream>>size and StandardFileStream>>position both invoke
primitives and return just fine.  Curious...  When I have more time I
will import my changes into another image and see if that helps, and
then I will try it on my Windows partition.

Thanks again.

=jason

On Mon, 06 May 2002 17:06:53 +0200, gaelli at emergent.de said:
> Jason Rogers <jacaetevha at fast-mail.org> schrieb am 06.05.2002,
> 16:29:15:
> > I have the need to count tokens within a string, including 'empty'
> > tokens.  Currently that capability doesn't exist (String implements
> > counting tokens but _skips_ empty ones).
> > 
> > Being a (relative) Smalltalk newbie, I would like daggers thrown at the
> > following code.  
> 
> OK, here they come... ;-)
> 
> Why don't you just count the number of delimters of each line and add
> 1?
> 
> ('a,b;c,c,,' select: [:aChar| ',;' includes: aChar]) size + 1  ---> 6
> 
> (If you only had one kind of delimiter, you could even write something
> like
> ('a,b,c,c,,' occurrencesOf: $,) + 1 ---> 6
> 
> Did I miss something?
> 
> Best,
> 
> Markus
> 
> My gut tells me that there must be a better way, but I
> > don't quite know what that way is.
> > 
> > Thanks in advance,
> > =jason
> > 
> > code snippet:
> > "countTokens: aString
> > 
> > 	"Answers the number of tokens in this string, including 'empty' tokens.  A non-empty string with the absence of a delimeter returns a total of 1 (one) token."
> > 
> > 	| startIndex counter |
> > 	(aString isEmpty) ifTrue:[^0].
> > 	startIndex _ 1.
> > 	counter _ 0.
> > 	[(startIndex  0]]
> > 		whileTrue: [
> > 			startIndex _ aString findString: Delimeter startingAt: (startIndex + 1).
> > 			counter _ counter + 1.
> > 		].
> > 	^ counter."
> > -- 
> >   Jason Rogers
> >   jacaetevha at fast-mail.org
> 
> 

-- 
  Jason Rogers
  jacaetevha at fast-mail.org



More information about the Squeak-dev mailing list