ANSI Smalltalk streams
Richard A. O'Keefe
ok at cs.otago.ac.nz
Mon Mar 3 04:18:21 UTC 2003
Before you say "take this to comp.lang.smalltalk", I don't get
comp.lang.smalltalk, and I'm interested in the relationship between
ANSI Smalltalk and Squeak, not other Smalltalks.
The thing which strikes me forcibly is how few changes there are between
the freely available 1.9 draft and the official standard. This includes
very little in the way of typo correction...
Here's a list of issues I have found in sections 5.9 "Stream Protocols"
and 5.10 "File Stream Protocols". Note the word "ISSUE"; this is not the
same as "ERROR".
1. "Some stream classes will build sequenceable collections or report
the values of a sequenceable collection." p255
These are the only occurrences of "sequenceable collection";
SequenceableCollection is not an ANSI class or protocol and
there is no "sequenceable collection" concept elsewhere in the ANSI ST
2. "Other types of streams may operate on files, positive integers,
random numbers, and so forth." p255
There _are_ ANSI Smalltalk streams that operate on files, but not
any that operate on random numbers. That's a pity, because most
Smalltalk users will not be expert in random number generation.
SCOPE issue; the standardisers no doubt had good reason for keeping
RNG out and there are public algorithms one can adapt. However, a
common protocol one could follow would have been nice. In particular,
should one ensure that 0.0 _is_ a possible result of #next or that
it is _not_?
3. "Transcript is a stream that may be used to log textual message[s]
generated by a Smalltalk program." p255
4. The protocol hierarchy lumps "gettable streams" (basically, things that
can support #atEnd and #next) and "peekable streams" (basically, things
that can in addition support #peek) into one <gettableStream> protocol.
As a means of specifying the streams that happen to be present in ANSI
Smalltalk itself, that's OK. But it provides people with no guidance
about which methods they should implement if they _can't_ implement #peek
(efficiently). This matters to me because I have several times been in
just that situation. #atEnd and #next, no problem. #peek, only if I
implemented lookahead myself (an extra two instance variables
'peeked lookahead' and slowdown in #atEnd and #next checking/maintaining
SCOPE. It is important for readers of the ANSI protocols to realise
that they are primarily meant for specifying ANSI classes; if they are
useful as models for your own classes, that's nice, but don't expect it.
5. 126.96.36.199 Definition, p257
"move objects in sequence from the front of the receiver's
future sequence values to the back of th[e] receiver's past..."
This occurs twice. Surely a spelling checker would have found
"the" written as "th"???
6. 188.8.131.52 Errors, p257
"If the receiver has any sequence values and amount is greater
than or equal to the total number of sequence values of the receiver".
But this forbids you moving the position to the end of the sequence.
I think this should read "If amount is greater than the total number
of sequence values of the receiver".
- 184.108.40.206 Definition
I find "appended with" grating. Less contentiously, the two
sentences are different in mood, for no apparent reason. Better as
"Sets the receiver's future sequence values to the concatenation of
its past sequence values and its future sequence values, and makes the
receiver's past sequence values empty."
- #contents and #upTo: are there, why not #upToEnd?
The irony here is that #upToEnd is implementable even in some
streams where #contents is not. But we've had discussions about
whether the standardisers were willing to require implementors to
add even quite tiny methods, and the answer was that they weren't.
This is an issue for me trying to put together an ANSI-like implementation
I could use, but the standard is what the standardisers wanted.
7. 220.127.116.11 page 259
"and, passed as the argument to an evaluation of operand."
The comma should not be there, and the argument is called
"operation" not "operand", as the very next sentence points out.
8. 18.104.22.168 #next: p259
"The result is undefined if amount is larger than the number of objects
in the receiver's future sequence values".
It really is not clear to me why the last block in a sequence of #next:
calls could not be short. (That's how read(2) works in UNIX, after all.)
This prompted me to look at Squeak. I find that in Squeak *some*
implementations of #next: will return a short block and some will go
rumble-rumble-CRASH. But it is easy to implement safely:
result := self species new: amount.
1 to: amount do: [:index |
self atEnd ifTrue: [^result copyFrom: 1 to: index - 1].
result at: index put: self next].
Why this matters is that if you have a stream that is not positionable,
you don't KNOW whether there are that many items left. As it is, the
ANSI definition of #next: pushes the responsibility on the user without
providing the user with any means of carrying out that responsibility,
other that to program his/her own version of #next:.
This is particularly important because some ANSI Smalltalk streams
are supposed to be positionable, but if you try it, you may be in for
a nasty surprise. (See below.)
The "style", so to speak, is a little inconsistent here, because
#upTo: *will* accept a final block that doesn't end in the usual way.
I'm not quite sure how to classify this, but a method you cannot use
when you most need it (see below) sounds like an issue to me.
9. 22.214.171.124 p260
"The results are undefined if there are no future sequence values
in the receiver." But there is only one result, a <boolean>.
10. 126.96.36.199 p264
"Has the effect of enumerating the (sic.) aCollection with the
message #do: and adding each element to the receiver with #nextPut:."
(a) The second "the" shouldn't be there.
(b) Collection methods generally say that elements are enumerated
_in the same order as_ they would be by #do: but don't actually
require that #do: be used. Squeak's implementations of #nextPutAll:
do *not* always use #do:, sometimes they use a block copy and it is
a pity that the standard doesn't actually let them do this.
(The difference is observable in Squeak.)
11. 5.9.10 <ReadWriteStream factory> Description p267
"<ReadWriteStreamfactory> provides for the creation of objects
conforming to the <WriteStream> protocol whose sequence values
are supplied by a collection."
(1) "ReadWriteStreamfactory" should be "ReadWriteStream factory"
(2) "WriteStream" should be "ReadWriteStream"
(3) "objects conforming to the ... protocol whose ... values ..."
"objects, conforming to the ... protocol, whose ... values ..."
- ReadStreams are created with #on:.
WriteStreams are created with #with:.
Why is there no #on: for ReadWriteStreams? You can get the same
effect by using (ReadWriteStream with: aCollection) reset; yourself
but why should you have to?
12. 5.10 pervasive
This is possibly the single biggest problem. Explicit reference is
made to POSIX, yet the strong assumption is made that every nameable
file is positionable.
To that I can only say '/dev/tty'.
(Well, I can also say pipes, named pipes. On my machine, there's
/dev/mouse and /dev/pty*. on Linux machines there's /dev/proc.
Unix systems have lots of unseekable "files". Rather strangely,
when I tested lseek() on /dev/tty or stdin, on one UNIX system it
claimed to have succeeded, and on another it closed my connection.
(Yes, really!) I think some people have interpreted EISPIPE too
It's not just a matter of #position and #position: being in the
interface, it's a matter of there being no provision at all for
Oh yes, there's another way that positioning can fail. Consider
VM/CMS. (My knowledge of VM/CMS ends with major version 6, and is
a little fuzzy because I haven't used it for a while.) VM/CMS is
record-oriented. It has fixed length records and variable length
records. Unusually, if you have a disc file full of variable length
records, you _can_ seek to an arbitrary record in the file. What
you can't do is see to an arbitrary byte/character. The only way I
can see to implement #position: on such files (which are quite a nice
way to represent text) is
self skip: amount.
which isn't very nice.
Then there is a third way that positioning can fail, and I'm surprised
that no-one who spent much time working on Xerox systems managed to get
the point across. It was certainly an issue in Xerox Quintus Prolog,
and the reason why I fought hard to keep byte-addressing for files
out of the Prolog standard. Consider a file of 16-bit text (our
original concern was the XNS 16-bit character set used in Xerox Lisp)
or wider (such as Unicode), represented in "compressed" form (these
days, think of UTF-8 or better still, UTR-6). With UTF-8, position
_as measured in characters_ is not the same as position _as measured
in bytes_, a problem that already exists in MS-DOS and Windows when
you do CRLF->cr mapping. (That's why the C standard does not require
byte addressing for text streams.) The simplest way out then is to
define #position to return a magic cookie that increases monotonically
with position in the file (as C does) and to define #position: to work
only for magic cookies recorded by #position, or possibly with 0.
But it gets worse. The "compressed" form used with XNS characters,
the SBCS->DBCS/DBCS->SBCS shift codes commonly used in VM/CMS, the
similar code set escapes used in ISO 2022, and the rather nice UTR-6
have the property that you cannot interpret bytes at some position in
a file without knowing what "shift state" you are in, even if the
position is known to be where a valid character starts. So a "position"
has to be an object that may contain a shift state as well as a byte
position. Indeed, it could encode position in character sequence,
position in underlying byte sequence, and shift state (if any).
The CRLF->cr problem is a serious one even for the file streams in
ANSI Smalltalk. A further problem here is that #cr is defined to
deposit the same sequence of characters for all streams, so that
aStream cr is not the same as aStream nextPut: Character cr.
Since Character cr is a single character, an ANSI Smalltalk implementation
appears to be *forbidden* to convert it to CR+LF on writing.
13. Also, there seems to be an assumption either that every file that
can be opened for output can be opened in read-write mode.
Well, in POSIX it is quite possible for a user to have permission
to write to a file without having permission to read from it. So
suppose there is a log file that I am supposed to append to:
logStream := FileStream write: logFileName mode: #'append'
check: true type: #'text'.
This makes sense, does it not? But #contents is in the <writeFileStream>
protocol, and it is absolutely unimplementable in this case, because
the file system permissions flatly refuse to let me see the "past
sequence values" of the file, yet there is no provision for #contents
failing. Nor is there any way for a program to determine whether it
_would_ fail without trying it and catching the entirely unspecified
error that might (or then again, might not) be raised.
There really _really_ need to be methods
canChangePosition -> <boolean>
canReportContents -> <boolean>
Ah, you say, if someone is writing ANSI Smalltalk, it is up to them
to ensure that they stay within the limits. But there is no way for
a programmer to find out whether they are staying within the limits
or not. Suppose you open a file (which _can_ be positioned), read a
file name from it, and try to open _that_ file. You have no way of
telling whether #position: or #contents can work, and in the case of
a file with '-w--w----' permission, no way of telling even whether the
attempt to open it will work, short of trying.
Failure to adequately consider current file systems.
14. 5.10.1 Description p270
"translated from or two" should be "translated from or to"
15. "is treated as a sequenced of 8-bit characters"
^ lose the "d"
16. 188.8.131.52 #next: Definition p271
"The result is undefined if amount is larger than the number of objects
in the receiver's future sequence values."
Remember that we're talking about reading from file system objects here.
The only way to know in advance whether there are that many characters
left is to ask whether
theStream contents size - theStream position >= amount
but because #contents is unimplementable for many file system objects,
the programmer cannot use this. This is not a cheap way to find the
size of a file especially in a 64-bit file system. So you might try
t := theStream position.
size := theStream position - t.
and cache the size. (It's a little odd that there is no #size message
for positionable streams so that this dance can be avoided.) However,
in many systems, the size of a file may change, even be reduced.
So between the time that you determine the amount of data left and
the time that you ask for the next n items, the amount left may have
changed, so even in a positionable stream, you CAN'T predict whether
it will be safe to call #next:.
So in the very case where #next: has the greatest payoff (from using
block copies) it is least safe to use it.
More information about the Squeak-dev