A new version of WebClient-Core was added to project The Inbox: http://source.squeak.org/inbox/WebClient-Core-monty.114.mcz
==================== Summary ====================
Name: WebClient-Core-monty.114 Author: monty Time: 30 October 2017, 2:59:47.866416 am UUID: 621a8ad8-0653-4117-a36e-28ef18c0e665 Ancestors: WebClient-Core-monty.113
Added support for disabling the automatic character decoding of text responses by adding #getContentWithProgress:decodesText:, making #getContentWithProgress: delegate to it, adding #contentWithProgress:decodesText: as a caching front-end and then making #contentWithProgress: delegate to it and #content delegate to #contentWithProgress:.
This is needed because some text formats allow encodings to be inferred from the document itself, like from a leading BOM or an XML-style 'encoding="..."?>' declaration, and to give an alternative for situations where the charset is incorrectly inferred and WebClient mangles the response. This is different from HTTP encoding (like with GZIP) which is part of HTTP and is meant to be applied before transit and removed after entirely transparant to the user and so does not permanently alter the original payload, unlike automatic decoding from UTF-8 or another character encoding.
=============== Diff against WebClient-Core-monty.113 ===============
Item was changed: ----- Method: WebMessage>>content (in category 'accessing') ----- content "Reads and caches available content and returns it."
+ ^ self contentWithProgress: nil - ^content ifNil:[content := self getContent]. !
Item was changed: ----- Method: WebMessage>>contentWithProgress: (in category 'accessing') ----- + contentWithProgress: progressBlockOrNil + "Reads and caches available content and returns it, periodically evaluating + progressBlockOrNil with the total size and the total bytes read so far." - contentWithProgress: progressBlock - "Reads and caches available content and returns it."
+ ^ self + contentWithProgress: progressBlockOrNil + decodesText: true! - ^content ifNil: [ content := self getContentWithProgress: progressBlock ]!
Item was added: + ----- Method: WebMessage>>contentWithProgress:decodesText: (in category 'accessing') ----- + contentWithProgress: progressBlockOrNil decodesText: aBoolean + "Reads and caches available content and returns it, periodically evaluating + progressBlockOrNil with the total size and the total bytes read so far, optionally + decoding text content based on the Content-Type charset." + + ^ content ifNil: [ + content := + self + getContentWithProgress: progressBlockOrNil + decodesText: aBoolean]!
Item was changed: ----- Method: WebMessage>>getContentWithProgress: (in category 'private') ----- getContentWithProgress: progressBlockOrNil + ^ self + getContentWithProgress: progressBlockOrNil + decodesText: true! - "Reads available content and returns it." - - | length result | - length := self contentLength. - result := (stream isBinary ifTrue:[ ByteArray ] ifFalse: [ ByteString ]) - new: (length ifNil: [ 1000 ]) - streamContents: [ :outputStream | - self - streamFrom: stream - to: outputStream - size: length - progress: progressBlockOrNil ]. - self decoderForContentEncoding ifNotNil: [:decoder | - result := (decoder on: result) upToEnd]. - self textConverterForContentType ifNotNil: [:converter | - [result := result convertFromWithConverter: converter] - on: InvalidUTF8 "some servers lie" - do: [^ result]]. - ^ result - !
Item was added: + ----- Method: WebMessage>>getContentWithProgress:decodesText: (in category 'private') ----- + getContentWithProgress: progressBlockOrNil decodesText: aBoolean + "Reads available content and returns it, periodically evaluating progressBlockOrNil + with the total size and the total bytes read so far, optionally decoding text content + based on the Content-Type charset." + + | length result | + length := self contentLength. + result := (stream isBinary ifTrue:[ ByteArray ] ifFalse: [ ByteString ]) + new: (length ifNil: [ 1000 ]) + streamContents: [ :outputStream | + self + streamFrom: stream + to: outputStream + size: length + progress: progressBlockOrNil ]. + self decoderForContentEncoding ifNotNil: [:decoder | + result := (decoder on: result) upToEnd]. + aBoolean ifTrue: [ + self textConverterForContentType ifNotNil: [:converter | + [result := result convertFromWithConverter: converter] + on: InvalidUTF8 "some servers lie" + do: [^ result]]]. + ^ result!
Item was removed: - ----- Method: WebResponse>>getContentWithProgress: (in category 'private') ----- - getContentWithProgress: progressBlockOrNil - | result | - - "Do not read any content if this was a HEAD request or code is 204 (no content)" - result := - (request method = 'HEAD' or: [code = 204]) - ifTrue: [''] - ifFalse: [super getContentWithProgress: progressBlockOrNil]. - "Reimplemented to close the socket if the request is transient" - self closeIfTransient. - ^ result.!
Item was added: + ----- Method: WebResponse>>getContentWithProgress:decodesText: (in category 'private') ----- + getContentWithProgress: progressBlockOrNil decodesText: aBoolean + | result | + + "Do not read any content if this was a HEAD request or code is 204 (no content)" + result := + (request method = 'HEAD' or: [code = 204]) + ifTrue: [''] + ifFalse: [ + super + getContentWithProgress: progressBlockOrNil + decodesText: aBoolean]. + "Reimplemented to close the socket if the request is transient" + self closeIfTransient. + ^ result.!