Christoph Thiede uploaded a new version of Collections to project The Trunk: http://source.squeak.org/trunk/Collections-ct.1041.mcz
==================== Summary ====================
Name: Collections-ct.1041 Author: ct Time: 1 June 2023, 9:40:33.122127 pm UUID: b55600ad-c59d-5840-b2fb-398b2377c8e0 Ancestors: Collections-mt.1040
Merges base32.4.cs: Adds support for base32 encoding and decoding. Adds Base32MimeConverter class and subsumes it with existing Base64MimeConverter class under new abstract RFC4648MimeConverter class. Adds #base32Encoded and #base32Decoded on String and String+ByteArray, resp. Adds convenient MimeConverter class>>#mimeDecode:. Tests new converter in new Base32MimeConverterTest and subsumes it with existing Base64MimeConverterTest under new abstract RFC4648MimeConverterTest.
Revision: Minor commentary improvements.
=============== Diff against Collections-mt.1040 ===============
Item was added: + RFC4648MimeConverter subclass: #Base32MimeConverter + instanceVariableNames: '' + classVariableNames: 'FromCharTable ToCharTable' + poolDictionaries: '' + category: 'Collections-Streams'! + + !Base32MimeConverter commentStamp: 'ct 5/18/2023 20:28' prior: 0! + I encode and decode data in base32 format. I translate a whole stream at once, taking a Stream as input and giving one as output, and answer a whole stream for the sender to use. + 0 A 8 I 16 Q 24 Y + 1 B 9 J 17 R 25 Z + 2 C 10 K 18 S 26 2 + 3 D 11 L 19 T 27 3 + 4 E 12 M 20 U 28 4 + 5 F 13 N 21 V 29 5 + 6 G 14 O 22 W 30 6 + 7 H 15 P 23 X 31 7 + (pad) = + Outbound: bytes are broken into 5 bit chunks, and the 0-31 value is converted to a character. 5 data bytes go into 8 characters. + Inbound: Characters are translated in to 0-31 values and shifted into 8 bit bytes.!
Item was added: + ----- Method: Base32MimeConverter class>>initialize (in category 'class initialization') ----- + initialize + + FromCharTable := Array new: 256. "nils" + ToCharTable := ($A to: $Z) , ($2 to: $7) , '='. + ToCharTable keysAndValuesDo: [:ind :char | + FromCharTable at: char asciiValue + 1 put: ind - 1].!
Item was added: + ----- Method: Base32MimeConverter>>fromCharTable (in category 'private') ----- + fromCharTable + + ^ FromCharTable!
Item was added: + ----- Method: Base32MimeConverter>>mimeDecode (in category 'conversion') ----- + mimeDecode + "Convert the stream in base 32 with only A-Z,2-7 to a full byte stream of characters. Answer a whole stream for the user to read." + + | nibA nibB nibC nibD nibE nibF nibG nibH | + [mimeStream atEnd] whileFalse: [ + (nibA := self nextValue) ifNil: [^ dataStream]. + (nibB := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibA bitShift: 3) + (nibB bitShift: -2)) asCharacter. + nibB := nibB bitAnd: 16r3. + (nibC := self nextValue) ifNil: [^ dataStream]. + (nibD := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibB bitShift: 6) + (nibC bitShift: 1) + (nibD bitShift: -4)) asCharacter. + nibD := nibD bitAnd: 16rF. + (nibE := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibD bitShift: 4) + (nibE bitShift: -1)) asCharacter. + nibE := nibE bitAnd: 16r1. + (nibF := self nextValue) ifNil: [^ dataStream]. + (nibG := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibE bitShift: 7) + (nibF bitShift: 2) + (nibG bitShift: -3)) asCharacter. + nibG := nibG bitAnd: 16r7. + (nibH := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibG bitShift: 5) + nibH) asCharacter. + ]. + ^ dataStream!
Item was added: + ----- Method: Base32MimeConverter>>mimeDecodeToByteArray (in category 'conversion') ----- + mimeDecodeToByteArray + "Convert the stream in base 32 with only A-Z,2-7 to a full ByteArray of 0-255 values. Answer a whole stream for the user to read." + + | nibA nibB nibC nibD nibE nibF nibG nibH | + [mimeStream atEnd] whileFalse: [ + (nibA := self nextValue) ifNil: [^ dataStream]. + (nibB := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibA bitShift: 3) + (nibB bitShift: -2)). + nibB := nibB bitAnd: 16r3. + (nibC := self nextValue) ifNil: [^ dataStream]. + (nibD := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibB bitShift: 6) + (nibC bitShift: 1) + (nibD bitShift: -4)). + nibD := nibD bitAnd: 16rF. + (nibE := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibD bitShift: 4) + (nibE bitShift: -1)). + nibE := nibE bitAnd: 16r1. + (nibF := self nextValue) ifNil: [^ dataStream]. + (nibG := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibE bitShift: 7) + (nibF bitShift: 2) + (nibG bitShift: -3)). + nibG := nibG bitAnd: 16r7. + (nibH := self nextValue) ifNil: [^ dataStream]. + dataStream nextPut: ((nibG bitShift: 5) + nibH). + ]. + ^ dataStream!
Item was added: + ----- Method: Base32MimeConverter>>mimeEncode (in category 'conversion') ----- + mimeEncode + "Convert from data to 5 bit characters." + + | padding raw nib lineLength | + padding := nil. + lineLength := 0. + [dataStream atEnd] whileFalse: [ + (multiLine and:[lineLength >= 70]) ifTrue: [ mimeStream cr. lineLength := 0. ]. + data := raw := dataStream next asInteger. + nib := (data bitAnd: 16rF8) bitShift: -3. "1111 1000" + mimeStream nextPut: (ToCharTable at: nib+1). + (raw := dataStream next) ifNil: [raw := 0. padding := -6]. + data := ((data bitAnd: 7) bitShift: 8) + raw asInteger. "0000 0111" + nib := (data bitAnd: 16r7C0) bitShift: -6. "0000 0111 1100 0000" + mimeStream nextPut: (ToCharTable at: nib+1). + nib := (data bitAnd: 16r3E) bitShift: -1. "0011 1110" + mimeStream nextPut: (ToCharTable at: nib+1). + (raw := dataStream next) ifNil: [raw := 0. padding ifNil: [padding := -4]]. + data := ((data bitAnd: 1) bitShift: 8) + (raw asInteger). "0000 0001" + nib := (data bitAnd: 16r1F0) bitShift: -4. "0000 0001 1111 0000" + mimeStream nextPut: (ToCharTable at: nib+1). + (raw := dataStream next) ifNil: [raw := 0. padding ifNil: [padding := -3]]. + data := ((data bitAnd: 16rF) bitShift: 8) + (raw asInteger). "0000 1111" + nib := (data bitAnd: 16rF80) bitShift: -7. "0000 1111 1000 0000" + mimeStream nextPut: (ToCharTable at: nib+1). + nib := (data bitAnd: 16r7C) bitShift: -2. "0111 1100" + mimeStream nextPut: (ToCharTable at: nib+1). + (raw := dataStream next) ifNil: [raw := 0. padding ifNil: [padding := -1]]. + data := ((data bitAnd: 3) bitShift: 8) + (raw asInteger). "0000 0011" + nib := (data bitAnd: 16r3E0) bitShift: -5. "0000 0011 1110 0000" + mimeStream nextPut: (ToCharTable at: nib+1). + nib := (data bitAnd: 16r1F). "0001 1111" + mimeStream nextPut: (ToCharTable at: nib+1). + + lineLength := lineLength + 8.]. + + padding ifNotNil: [ + mimeStream skip: padding. + padding to: -1 do: [:i | mimeStream nextPut: $=]. + ^ mimeStream].!
Item was changed: + RFC4648MimeConverter subclass: #Base64MimeConverter + instanceVariableNames: '' - MimeConverter subclass: #Base64MimeConverter - instanceVariableNames: 'data multiLine' classVariableNames: 'FromCharTable ToCharTable' poolDictionaries: '' category: 'Collections-Streams'!
!Base64MimeConverter commentStamp: '<historical>' prior: 0! This class encodes and decodes data in Base64 format. This is MIME encoding. We translate a whole stream at once, taking a Stream as input and giving one as output. Returns a whole stream for the caller to use. 0 A 17 R 34 i 51 z 1 B 18 S 35 j 52 0 2 C 19 T 36 k 53 1 3 D 20 U 37 l 54 2 4 E 21 V 38 m 55 3 5 F 22 W 39 n 56 4 6 G 23 X 40 o 57 5 7 H 24 Y 41 p 58 6 8 I 25 Z 42 q 59 7 9 J 26 a 43 r 60 8 10 K 27 b 44 s 61 9 11 L 28 c 45 t 62 + 12 M 29 d 46 u 63 / 13 N 30 e 47 v 14 O 31 f 48 w (pad) = 15 P 32 g 49 x 16 Q 33 h 50 y Outbound: bytes are broken into 6 bit chunks, and the 0-63 value is converted to a character. 3 data bytes go into 4 characters. Inbound: Characters are translated in to 0-63 values and shifted into 8 bit bytes.
(See: N. Borenstein, Bellcore, N. Freed, Innosoft, Network Working Group, Request for Comments: RFC 1521, September 1993, MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies. Sec 6.2)
By Ted Kaehler, based on Tim Olson's Base64Filter.!
Item was removed: - ----- Method: Base64MimeConverter class>>decodeInteger: (in category 'convenience') ----- - decodeInteger: mimeString - | bytes sum | - "Decode the MIME string into an integer of any length" - - bytes := (Base64MimeConverter mimeDecodeToBytes: - (ReadStream on: mimeString)) contents. - sum := 0. - bytes reverseDo: [:by | sum := sum * 256 + by]. - ^ sum!
Item was removed: - ----- Method: Base64MimeConverter class>>encodeInteger: (in category 'convenience') ----- - encodeInteger: int - | strm | - "Encode an integer of any length and return the MIME string" - - strm := WriteStream on: (ByteArray new: int digitLength). - 1 to: int digitLength do: [:ii | strm nextPut: (int digitAt: ii)]. - ^ ((self mimeEncode: strm readStream) contents) copyUpTo: $= "remove padding"!
Item was removed: - ----- Method: Base64MimeConverter class>>mimeDecodeToBytes: (in category 'convenience') ----- - mimeDecodeToBytes: aStream - "Return a ReadStream of the original ByteArray. aStream has only 65 innocuous character values. aStream is not binary. (See class comment). 4 bytes in aStream goes to 3 bytes in output." - - | me | - aStream position: 0. - me := self new mimeStream: aStream. - me dataStream: (WriteStream on: (ByteArray new: aStream size * 3 // 4)). - me mimeDecodeToByteArray. - ^ me dataStream readStream!
Item was removed: - ----- Method: Base64MimeConverter class>>mimeDecodeToChars: (in category 'convenience') ----- - mimeDecodeToChars: aStream - "Return a ReadWriteStream of the original String. aStream has only 65 innocuous character values. It is not binary. (See class comment). 4 bytes in aStream goes to 3 bytes in output." - - | me | - aStream position: 0. - me := self new mimeStream: aStream. - me dataStream: (WriteStream on: (String new: aStream size * 3 // 4)). - me mimeDecode. - ^ me dataStream readStream!
Item was removed: - ----- Method: Base64MimeConverter class>>mimeEncode: (in category 'convenience') ----- - mimeEncode: aStream - "Return a ReadWriteStream of characters. The data of aStream is encoded as 65 innocuous characters. (See class comment). 3 bytes in aStream goes to 4 bytes in output." - ^self mimeEncode: aStream multiLine: true atStart: true!
Item was removed: - ----- Method: Base64MimeConverter class>>mimeEncode:multiLine: (in category 'convenience') ----- - mimeEncode: aStream multiLine: aBool - "Return a ReadWriteStream of characters. The data of aStream is encoded as 65 innocuous characters. (See class comment). 3 bytes in aStream goes to 4 bytes in output." - - ^self mimeEncode: aStream multiLine: aBool atStart: true!
Item was removed: - ----- Method: Base64MimeConverter class>>mimeEncode:multiLine:atStart: (in category 'private - convenience') ----- - mimeEncode: aStream multiLine: aBool atStart: resetInput - "Return a ReadStream of characters. The data of aStream is encoded as 65 innocuous characters. (See class comment). 3 bytes in aStream goes to 4 bytes in output." - - | me | - resetInput ifTrue:[aStream position: 0]. - me := self new dataStream: aStream. - me multiLine: aBool. - me mimeStream: (WriteStream on: (String new: aStream size + 20 * 4 // 3)). - me mimeEncode. - ^ me mimeStream readStream!
Item was removed: - ----- Method: Base64MimeConverter class>>mimeEncodeContinue: (in category 'private - convenience') ----- - mimeEncodeContinue: aStream - "Return a ReadWriteStream of characters. The data of aStream is encoded as 65 innocuous characters. (See class comment). 3 bytes in aStream goes to 4 bytes in output." - ^self mimeEncode: aStream multiLine: true atStart: false!
Item was added: + ----- Method: Base64MimeConverter>>fromCharTable (in category 'private') ----- + fromCharTable + + ^ FromCharTable!
Item was changed: ----- Method: Base64MimeConverter>>mimeDecode (in category 'conversion') ----- mimeDecode + "Convert the stream in base 64 with only a-z,A-Z,0-9,+,/ to a full byte stream of characters. Answer a whole stream for the user to read." - "Convert a stream in base 64 with only a-z,A-Z,0-9,+,/ to a full byte stream of characters. Return a whole stream for the user to read."
| nibA nibB nibC nibD | [mimeStream atEnd] whileFalse: [ (nibA := self nextValue) ifNil: [^ dataStream]. (nibB := self nextValue) ifNil: [^ dataStream]. dataStream nextPut: ((nibA bitShift: 2) + (nibB bitShift: -4)) asCharacter. nibB := nibB bitAnd: 16rF. (nibC := self nextValue) ifNil: [^ dataStream]. dataStream nextPut: ((nibB bitShift: 4) + (nibC bitShift: -2)) asCharacter. nibC := nibC bitAnd: 16r3. (nibD := self nextValue) ifNil: [^ dataStream]. dataStream nextPut: ((nibC bitShift: 6) + nibD) asCharacter. ]. ^ dataStream!
Item was changed: ----- Method: Base64MimeConverter>>mimeDecodeToByteArray (in category 'conversion') ----- mimeDecodeToByteArray + "Convert the stream in base 64 with only a-z,A-Z,0-9,+,/ to a full ByteArray of 0-255 values. Answer a whole stream for the user to read." - "Convert a stream in base 64 with only a-z,A-Z,0-9,+,/ to a full ByteArray of 0-255 values. Return a whole stream for the user to read."
| nibA nibB nibC nibD | [mimeStream atEnd] whileFalse: [ (nibA := self nextValue) ifNil: [^ dataStream]. (nibB := self nextValue) ifNil: [^ dataStream]. dataStream nextPut: ((nibA bitShift: 2) + (nibB bitShift: -4)). nibB := nibB bitAnd: 16rF. (nibC := self nextValue) ifNil: [^ dataStream]. dataStream nextPut: ((nibB bitShift: 4) + (nibC bitShift: -2)). nibC := nibC bitAnd: 16r3. (nibD := self nextValue) ifNil: [^ dataStream]. dataStream nextPut: ((nibC bitShift: 6) + nibD). ]. ^ dataStream!
Item was removed: - ----- Method: Base64MimeConverter>>multiLine (in category 'accessing') ----- - multiLine - "Determines whether we allow multi-line encodings (the default) or force everything into a single line (for use with URLs etc. where the continuation marker and the line break cause problems)" - ^multiLine!
Item was removed: - ----- Method: Base64MimeConverter>>multiLine: (in category 'accessing') ----- - multiLine: aBool - "Determines whether we allow multi-line encodings (the default) or force everything into a single line (for use with URLs etc. where the continuation marker and the line break cause problems)" - multiLine := aBool!
Item was removed: - ----- Method: Base64MimeConverter>>nextValue (in category 'conversion') ----- - nextValue - "The next six bits of data char from the mimeStream, or nil. Skip all other chars" - | raw num | - [raw := mimeStream next. - raw ifNil: [^ nil]. "end of stream" - raw == $= ifTrue: [^ nil]. - num := FromCharTable at: raw asciiValue + 1. - num ifNotNil: [^ num]. - "else ignore space, return, tab, ..." - ] repeat!
Item was added: + ----- Method: ByteArray>>base32Encoded (in category 'converting') ----- + base32Encoded + "Encode the receiver as base32" + "'Hello World' asByteArray base32Encoded" + ^(Base32MimeConverter mimeEncode: self readStream) contents!
Item was added: + ----- Method: MimeConverter class>>mimeDecode: (in category 'convenience') ----- + mimeDecode: aCollectionOrStream + + ^ String streamContents: [:out | + self mimeDecode: aCollectionOrStream to: out]!
Item was added: + MimeConverter subclass: #RFC4648MimeConverter + instanceVariableNames: 'data multiLine' + classVariableNames: '' + poolDictionaries: '' + category: 'Collections-Streams'! + + !RFC4648MimeConverter commentStamp: 'ct 5/18/2023 20:28' prior: 0! + I am the abstract superclass for power-of-two-base mime converters according to RFC4648. We translate a whole stream at once, taking a Stream as input and giving one as output, and answer a whole stream for the sender to use.!
Item was added: + ----- Method: RFC4648MimeConverter class>>decodeInteger: (in category 'convenience') ----- + decodeInteger: mimeString + "Decode the MIME string into an integer of any length" + + | bytes sum | + bytes := (self mimeDecodeToBytes: + (ReadStream on: mimeString)) contents. + sum := 0. + bytes reverseDo: [:by | sum := sum * 256 + by]. + ^ sum!
Item was added: + ----- Method: RFC4648MimeConverter class>>encodeInteger: (in category 'convenience') ----- + encodeInteger: anInteger + "Encode anInteger of any length and return the MIME string" + + | stream | + stream := WriteStream on: (ByteArray new: anInteger digitLength). + 1 to: anInteger digitLength do: [:ii | stream nextPut: (anInteger digitAt: ii)]. + ^ ((self mimeEncode: stream readStream) contents) copyUpTo: $= "remove padding"!
Item was added: + ----- Method: RFC4648MimeConverter class>>mimeDecodeToBytes: (in category 'convenience') ----- + mimeDecodeToBytes: aStream + "Answer a ReadStream of the original ByteArray. aStream has only 33/65/... innocuous character values. aStream is not binary. (See class comment). 8/4/... bytes in aStream goes to 5/3/... bytes in output." + + | me | + aStream position: 0. + me := self new mimeStream: aStream. + me dataStream: (WriteStream on: (ByteArray new: aStream size * 3 // 4)). + me mimeDecodeToByteArray. + ^ me dataStream readStream!
Item was added: + ----- Method: RFC4648MimeConverter class>>mimeDecodeToChars: (in category 'convenience') ----- + mimeDecodeToChars: aStream + "Answer a ReadWriteStream of the original String. aStream has only 33/65/... innocuous character values. It is not binary. (See class comment). 8/4/... bytes in aStream goes to 5/3/... bytes in output." + + | me | + aStream position: 0. + me := self new mimeStream: aStream. + me dataStream: (WriteStream on: (String new: aStream size * 3 // 4)). + me mimeDecode. + ^ me dataStream readStream!
Item was added: + ----- Method: RFC4648MimeConverter class>>mimeEncode: (in category 'convenience') ----- + mimeEncode: aStream + "Answer a ReadWriteStream of characters. The data of aStream is encoded as 33/65/... innocuous characters. (See class comment). 5/3/... bytes in aStream goes to 8/4/... bytes in output." + self flag: #suspicious. "ct: All other classes answer a String, just this one answers a ReadStream. Unfortunately, expectations of senders are inconsistent as well ..." + ^self mimeEncode: aStream multiLine: true atStart: true!
Item was added: + ----- Method: RFC4648MimeConverter class>>mimeEncode:multiLine: (in category 'convenience') ----- + mimeEncode: aStream multiLine: aBool + "Answer a ReadWriteStream of characters. The data of aStream is encoded as 33/65/... innocuous characters. (See class comment). 5/3/... bytes in aStream goes to 8/4/... bytes in output." + + ^self mimeEncode: aStream multiLine: aBool atStart: true!
Item was added: + ----- Method: RFC4648MimeConverter class>>mimeEncode:multiLine:atStart: (in category 'private - convenience') ----- + mimeEncode: aStream multiLine: aBool atStart: resetInput + "Answer a ReadStream of characters. The data of aStream is encoded as 33/65/... innocuous characters. (See class comment). 5/3/... bytes in aStream goes to 8/4/... bytes in output." + + | me | + resetInput ifTrue:[aStream position: 0]. + me := self new dataStream: aStream. + me multiLine: aBool. + me mimeStream: (WriteStream on: (String new: aStream size + 20 * 4 // 3)). + me mimeEncode. + ^ me mimeStream readStream!
Item was added: + ----- Method: RFC4648MimeConverter class>>mimeEncodeContinue: (in category 'private - convenience') ----- + mimeEncodeContinue: aStream + "Answer a ReadWriteStream of characters. The data of aStream is encoded as 33/65/... innocuous characters. (See class comment). 5/3/... bytes in aStream goes to 8/4/... bytes in output." + ^self mimeEncode: aStream multiLine: true atStart: false!
Item was added: + ----- Method: RFC4648MimeConverter>>fromCharTable (in category 'private') ----- + fromCharTable + + ^ self subclassResponsibility!
Item was added: + ----- Method: RFC4648MimeConverter>>mimeDecode (in category 'conversion') ----- + mimeDecode + "Convert the stream in the respective base to a full byte stream of characters. Answer a whole stream for the user to read." + + ^ self subclassResponsibility!
Item was added: + ----- Method: RFC4648MimeConverter>>mimeDecodeToByteArray (in category 'conversion') ----- + mimeDecodeToByteArray + "Convert the stream in the respective base to a full ByteArray of 0-255 values. Answer a whole stream for the user to read." + + ^ self subclassResponsibility!
Item was added: + ----- Method: RFC4648MimeConverter>>mimeEncode (in category 'conversion') ----- + mimeEncode + "Convert from data to n bit characters." + + ^ self subclassResponsibility!
Item was added: + ----- Method: RFC4648MimeConverter>>multiLine (in category 'accessing') ----- + multiLine + "Determines whether we allow multi-line encodings (the default) or force everything into a single line (for use with URLs etc. where the continuation marker and the line break cause problems)" + ^multiLine!
Item was added: + ----- Method: RFC4648MimeConverter>>multiLine: (in category 'accessing') ----- + multiLine: aBool + "Determines whether we allow multi-line encodings (the default) or force everything into a single line (for use with URLs etc. where the continuation marker and the line break cause problems)" + multiLine := aBool!
Item was added: + ----- Method: RFC4648MimeConverter>>nextValue (in category 'conversion') ----- + nextValue + "The next n bits of data char from the mimeStream, or nil. Skip all other chars" + | raw num | + [raw := mimeStream next. + raw ifNil: [^ nil]. "end of stream" + raw == $= ifTrue: [^ nil]. + num := self fromCharTable at: raw asciiValue + 1. + num ifNotNil: [^ num]. + "else ignore space, return, tab, ..." + ] repeat!
Item was added: + ----- Method: String>>base32Decoded (in category 'converting') ----- + base32Decoded + "Decode the receiver from base 32" + "'JBSWY3DPEBLW64TMMQ======' base32Decoded" + ^ Base32MimeConverter mimeDecode: self as: self class!
Item was added: + ----- Method: String>>base32Encoded (in category 'converting') ----- + base32Encoded + "Encode the receiver as base32" + "'Hello World' base32Encoded" + + ^(Base32MimeConverter + mimeEncode: (ReadStream on: self) + multiLine: false) contents!
packages@lists.squeakfoundation.org