[squeak-dev] The Inbox: Monticello-ul.727.mcz
Levente Uzonyi
leves at caesar.elte.hu
Sun Sep 20 19:36:29 UTC 2020
Hi Christoph,
Done. Along with Monticello-ct.726 and Monticello-ul.726.
Levente
On Sat, 19 Sep 2020, Thiede, Christoph wrote:
>
> Hi Levente,
>
>
> alright, then please go ahead, I'm looking forward to seeing this in the Trunk! :-)
>
>
> Best,
>
> Christoph
>
> _________________________________________________________________________________________________________________________________________________________________________________________________________________________________
> Von: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im Auftrag von Levente Uzonyi <leves at caesar.elte.hu>
> Gesendet: Donnerstag, 17. September 2020 12:49:28
> An: The general-purpose Squeak developers list
> Betreff: Re: [squeak-dev] The Inbox: Monticello-ul.727.mcz
> Hi Christoph,
>
> On Thu, 17 Sep 2020, Thiede, Christoph wrote:
>
> >
> > Very nice idea! :-) If I wanted to hijack an image, these URLRewriteRules would probably be my first approach - but security has never been an issue for Squeak, so I guess this is not a problem.
>
> If you can manipulate objects like URLRewriteRules, you have already
> hijacked the image.
>
> >
> >
> > Does this also fix the problem with the classic HTTP URLs returned by the update map or will we still need to patch them on the server side?
>
> The urls in the update map are used to create repositories. That's why
> simply changing the existing http repository urls to https doesn't
> suffice, because the updater will see the http urls and create new
> repositores with them if they are absent.
>
> So, this rewrite trick of Eliot works around the problem of the update
> maps as well.
>
> >
> >
> > And one last question regarding to your tests in the method comment of #rewriteUrl:forDownload:: Couldn't you put them into a real test case? I'm pretty sure that not everyone will run these out-commented tests manually,
> and
> > it would be a pity not to automate them.
>
> I left the possibility to change the rewrite rules to whatever you want.
> If there were a test case with those asserts in the comment, the test
> would start failing as soon as you changed the rules.
> Thought it's possible to create a test case which temporarily resets to
> the default rules and tests them.
>
>
> Levente
>
> >
> >
> > Best,
> >
> > Christoph
> >
> >________________________________________________________________________________________________________________________________________________________________________________________________________________________________
> _
> > Von: Squeak-dev <squeak-dev-bounces at lists.squeakfoundation.org> im Auftrag von commits at source.squeak.org <commits at source.squeak.org>
> > Gesendet: Donnerstag, 17. September 2020 01:55:41
> > An: squeak-dev at lists.squeakfoundation.org
> > Betreff: [squeak-dev] The Inbox: Monticello-ul.727.mcz
> > Levente Uzonyi uploaded a new version of Monticello to project The Inbox:
> > http://source.squeak.org/inbox/Monticello-ul.727.mcz
> >
> > ==================== Summary ====================
> >
> > Name: Monticello-ul.727
> > Author: ul
> > Time: 17 September 2020, 1:54:51.056164 am
> > UUID: ad776836-42eb-4aa2-b788-f10dd9e07da2
> > Ancestors: Monticello-cmm.726
> >
> > MCHttpRepository changes:
> > - before up- or downloading files, transform the urls using #rewriteUrl:forDownload:. The default rules (see #urlRewriteRules) switch from http to https for source.squeak.org and squeaksource.com, and switch to the the
> static
> > smalltalkhub site for downloads. The url rewriting is Eliot's idea, but this implementation uses a list of rewrite rules instead of a dictionary-based mapping.
> > - use WebClient (and the shared webclient instance) for uploads too
> > - retry down/uploading with WebClient at most 3 times. This should work around the case where the underlying socket was closed but the state of the socket has not been updated in Squeak.
> > - use https in #creationTemplate
> >
> > =============== Diff against Monticello-cmm.726 ===============
> >
> > Item was changed:
> > MCFileBasedRepository subclass: #MCHttpRepository
> > instanceVariableNames: 'location user password readerCache indexed webClient'
> > + classVariableNames: 'URLRewriteRules UseSharedWebClientInstance'
> > - classVariableNames: 'UseSharedWebClientInstance'
> > poolDictionaries: ''
> > category: 'Monticello-Repositories'!
> >
> > Item was changed:
> > ----- Method: MCHttpRepository class>>creationTemplate (in category 'ui-support') -----
> > creationTemplate
> > + ^self creationTemplateLocation: 'https://www.squeaksource.com/ProjectName'
> > - ^self creationTemplateLocation: 'http://www.squeaksource.com/ProjectName'
> > user: 'squeak'
> > password: 'squeak'
> > !
> >
> > Item was added:
> > + ----- Method: MCHttpRepository class>>rewriteUrl:forDownload: (in category 'url rewrite') -----
> > + rewriteUrl: aString forDownload: forDownload
> > +
> > + | result |
> > + result := aString.
> > + self urlRewriteRules groupsDo: [ :regexString :replacement :downloadOnly |
> > + (forDownload or: [ downloadOnly not ]) ifTrue: [
> > + result := result copyWithRegex: regexString matchesReplacedWith: replacement ] ].
> > + ^result
> > +
> > + "
> > + self assert: 'https://squeaksource.com/foo/bar?baz=1' = (self rewriteUrl: 'http://squeaksource.com/foo/bar?baz=1' forDownload: true).
> > + self assert: 'https://squeaksource.com/foo/bar?baz=1' = (self rewriteUrl: 'https://squeaksource.com/foo/bar?baz=1' forDownload: true).
> > + self assert: 'https://source.squeak.org/foo/bar?baz=1' = (self rewriteUrl: 'http://source.squeak.org/foo/bar?baz=1' forDownload: true).
> > + self assert: 'https://source.squeak.org/foo/bar?baz=1' = (self rewriteUrl: 'https://source.squeak.org/foo/bar?baz=1' forDownload: true).
> > + self assert: 'http://static.smalltalkhub.com/foo/bar?baz=1' = (self rewriteUrl: 'http://smalltalkhub.com/foo/bar?baz=1' forDownload: true).
> > + self assert: 'http://smalltalkhub.com/foo/bar?baz=1' = (self rewriteUrl: 'http://smalltalkhub.com/foo/bar?baz=1' forDownload: false).
> > + "!
> >
> > Item was added:
> > + ----- Method: MCHttpRepository class>>urlRewriteRules (in category 'url rewrite') -----
> > + urlRewriteRules
> > +
> > + ^URLRewriteRules ifNil: [
> > + URLRewriteRules := #(
> > + "Regex to be replaced" "static replacement string" "download only"
> > + '^http\://source\.squeak\.org/' 'https://source.squeak.org/' false
> > + '^http\://squeaksource\.com/' 'https://squeaksource.com/' false
> > + '^http\://www.squeaksource\.com/' 'https://www.squeaksource.com/' false
> > + '^http\://smalltalkhub.com/' 'http://static.smalltalkhub.com/' true
> > + ) asOrderedCollection ]!
> >
> > Item was changed:
> > ----- Method: MCHttpRepository>>httpGet:arguments: (in category 'private') -----
> > httpGet: url arguments: arguments
> >
> > + | urlString |
> > - | progress urlString client response result |
> > - progress := [ :total :amount |
> > - HTTPProgress new
> > - total: total;
> > - amount: amount;
> > - signal: 'Downloading...' ].
> > urlString := arguments
> > ifNil: [ url ]
> > ifNotNil: [
> > | queryString |
> > queryString := WebUtils encodeUrlEncodedForm: arguments.
> > (url includes: $?)
> > ifTrue: [ url, '&', queryString ]
> > ifFalse: [ url, '?', queryString ] ].
> > + urlString := self class rewriteUrl: urlString forDownload: true.
> > + ^self webClientDo: [ :client |
> > + client
> > + username: self user;
> > + password: self password;
> > + httpGet: urlString do: [ :request |
> > + request
> > + headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;
> > + headerAt: 'Connection' put: 'Keep-Alive';
> > + headerAt: 'Accept' put: '*/*' ] ]!
> > - self class useSharedWebClientInstance ifTrue: [
> > - "Acquire webClient by atomically storing it in the client variable and setting its value to nil."
> > - client := webClient.
> > - webClient := nil ].
> > - client
> > - ifNil: [ client := WebClient new ]
> > - ifNotNil: [
> > - "Attempt to avoid an error on windows by recreating the underlying stream."
> > - client isConnected ifFalse: [ client close ] ].
> > - response := client
> > - username: self user;
> > - password: self password;
> > - httpGet: urlString do: [ :request |
> > - request
> > - headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;
> > - headerAt: 'Connection' put: 'Keep-Alive';
> > - headerAt: 'Accept' put: '*/*' ].
> > - result := (response code between: 200 and: 299)
> > - ifFalse: [
> > - response content. "Make sure content is read."
> > - nil ]
> > - ifTrue: [ (RWBinaryOrTextStream with: (response contentWithProgress: progress)) reset ].
> > - self class useSharedWebClientInstance
> > - ifTrue: [
> > - "Save the WebClient instance for reuse, but only if there is no client cached."
> > - webClient
> > - ifNil: [ webClient := client ]
> > - ifNotNil: [ client close ] ]
> > - ifFalse: [ client close ].
> > - result ifNil: [ NetworkError signal: 'Could not access ', location ].
> > - ^result!
> >
> > Item was added:
> > + ----- Method: MCHttpRepository>>webClientDo: (in category 'private') -----
> > + webClientDo: aBlock
> > +
> > + | client attemptsLeft response result |
> > + self class useSharedWebClientInstance ifTrue: [
> > + "Acquire webClient by atomically storing it in the client variable and setting its value to nil."
> > + client := webClient.
> > + webClient := nil ].
> > +
> > + client
> > + ifNil: [ client := WebClient new ]
> > + ifNotNil: [
> > + "Attempt to avoid an error by recreating the underlying stream."
> > + client isConnected ifFalse: [ client close ] ].
> > +
> > + attemptsLeft := 3.
> > + response := nil.
> > + [ response isNil and: [ attemptsLeft > 0 ] ] whileTrue: [
> > + response := [ aBlock value: client ]
> > + on: NetworkError
> > + do: [ :error |
> > + attemptsLeft = 0 ifTrue: [ error pass ].
> > + (3 - attemptsLeft) seconds asDelay wait.
> > + attemptsLeft := attemptsLeft - 1.
> > + nil "The response" ] ].
> > +
> > + result := (response code between: 200 and: 299)
> > + ifFalse: [
> > + response content. "Make sure content is read."
> > + nil ]
> > + ifTrue: [
> > + (RWBinaryOrTextStream with: (
> > + response contentWithProgress: [ :total :amount |
> > + HTTPProgress new
> > + total: total;
> > + amount: amount;
> > + signal ])) reset ].
> > +
> > + self class useSharedWebClientInstance
> > + ifTrue: [
> > + "Save the WebClient instance for reuse, but only if there is no client cached."
> > + webClient
> > + ifNil: [ webClient := client ]
> > + ifNotNil: [ client close ] ]
> > + ifFalse: [ client close ].
> > +
> > + result ifNil: [ NetworkError signal: 'Could not access ', location ].
> > + ^result!
> >
> > Item was changed:
> > ----- Method: MCHttpRepository>>writeStreamForFileNamed:replace:do: (in category 'private') -----
> > writeStreamForFileNamed: aString replace: ignoreBoolean do: aBlock
> > +
> > + | stream urlString |
> > - | stream response statusLine code |
> > stream := RWBinaryOrTextStream on: String new.
> > aBlock value: stream.
> > + urlString := self urlForFileNamed: aString.
> > + urlString := self class rewriteUrl: urlString forDownload: false.
> > + ^self displayProgress: 'Uploading ', aString during: [
> > + self webClientDo: [ :client |
> > + client
> > + username: self user;
> > + password: self password;
> > + httpPut: urlString
> > + content: stream contents
> > + type: nil
> > + do: [ :request |
> > + request
> > + headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;
> > + headerAt: 'Connection' put: 'Keep-Alive';
> > + headerAt: 'Accept' put: '*/*' ] ] ]!
> > - self displayProgress: 'Uploading ', aString during:[
> > - response := HTTPSocket
> > - httpPut: stream contents
> > - to: (self urlForFileNamed: aString)
> > - user: self user
> > - passwd: self password.
> > - ].
> > - "More robust handling of HTTP responses. Instead of enumerating
> > - all possible return codes and http versions, do a quick parse"
> > - (response beginsWith: 'HTTP/') ifTrue:[
> > - "Looks like an HTTP header, not some error message"
> > - statusLine := response copyUpTo: Character cr.
> > - code := [(statusLine findTokens: ' ') second asInteger] on: Error do:[].
> > - ].
> > - (code isInteger and:[code between: 200 and: 299])
> > - ifFalse:[self error: response].!
> >
> >
> >
> >
>
>
More information about the Squeak-dev
mailing list
|