<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-15">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Hi Levente,</p>
<p><br>
</p>
<p>alright, then please go ahead, I'm looking forward to seeing this in the Trunk! :-)</p>
<p><br>
</p>
<p>Best,</p>
<p>Christoph</p>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div name="x_divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div>
<div class="x__rp_T4" id="x_Item.MessagePartBody">
<div class="x__rp_U4 x_ms-font-weight-regular x_ms-font-color-neutralDark x_rpHighlightAllClass x_rpHighlightBodyClass" id="x_Item.MessageUniqueBody" style="font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif,serif,EmojiFont">
<div dir="ltr">
<div id="x_divtagdefaultwrapper"><font face="Calibri,Helvetica,sans-serif,EmojiFont,Apple Color Emoji,Segoe UI Emoji,NotoColorEmoji,Segoe UI Symbol,Android Emoji,EmojiSymbols">
<div id="x_Signature">
<div style="margin:0px"><font style="font-family:Calibri,Arial,Helvetica,sans-serif,serif,EmojiFont">
<div><font size="3" color="black"><span style="font-size:12pt"><a href="http://www.hpi.de/" target="_blank" rel="noopener noreferrer" id="LPNoLP"><font size="2"><span id="LPlnk909538"><font color="#757B80"></font></span></font></a></span></font></div>
</font></div>
</div>
</font></div>
</div>
</div>
</div>
</div>
<div><font size="2" color="#808080"></font></div>
</div>
</div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Von:</b> Squeak-dev <squeak-dev-bounces@lists.squeakfoundation.org> im Auftrag von Levente Uzonyi <leves@caesar.elte.hu><br>
<b>Gesendet:</b> Donnerstag, 17. September 2020 12:49:28<br>
<b>An:</b> The general-purpose Squeak developers list<br>
<b>Betreff:</b> Re: [squeak-dev] The Inbox: Monticello-ul.727.mcz</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi Christoph,<br>
<br>
On Thu, 17 Sep 2020, Thiede, Christoph wrote:<br>
<br>
> <br>
> Very nice idea! :-) If I wanted to hijack an image, these URLRewriteRules would probably be my first approach - but security has never been an issue for Squeak, so I guess this is not a problem.<br>
<br>
If you can manipulate objects like URLRewriteRules, you have already <br>
hijacked the image.<br>
<br>
> <br>
> <br>
> Does this also fix the problem with the classic HTTP URLs returned by the update map or will we still need to patch them on the server side?<br>
<br>
The urls in the update map are used to create repositories. That's why <br>
simply changing the existing http repository urls to https doesn't <br>
suffice, because the updater will see the http urls and create new <br>
repositores with them if they are absent.<br>
<br>
So, this rewrite trick of Eliot works around the problem of the update <br>
maps as well.<br>
<br>
> <br>
> <br>
> And one last question regarding to your tests in the method comment of #rewriteUrl:forDownload:: Couldn't you put them into a real test case? I'm pretty sure that not everyone will run these out-commented tests manually, and<br>
> it would be a pity not to automate them.<br>
<br>
I left the possibility to change the rewrite rules to whatever you want. <br>
If there were a test case with those asserts in the comment, the test <br>
would start failing as soon as you changed the rules.<br>
Thought it's possible to create a test case which temporarily resets to <br>
the default rules and tests them.<br>
<br>
<br>
Levente<br>
<br>
> <br>
> <br>
> Best,<br>
> <br>
> Christoph<br>
> <br>
> _________________________________________________________________________________________________________________________________________________________________________________________________________________________________<br>
> Von: Squeak-dev <squeak-dev-bounces@lists.squeakfoundation.org> im Auftrag von commits@source.squeak.org <commits@source.squeak.org><br>
> Gesendet: Donnerstag, 17. September 2020 01:55:41<br>
> An: squeak-dev@lists.squeakfoundation.org<br>
> Betreff: [squeak-dev] The Inbox: Monticello-ul.727.mcz <br>
> Levente Uzonyi uploaded a new version of Monticello to project The Inbox:<br>
> <a href="http://source.squeak.org/inbox/Monticello-ul.727.mcz">http://source.squeak.org/inbox/Monticello-ul.727.mcz</a><br>
> <br>
> ==================== Summary ====================<br>
> <br>
> Name: Monticello-ul.727<br>
> Author: ul<br>
> Time: 17 September 2020, 1:54:51.056164 am<br>
> UUID: ad776836-42eb-4aa2-b788-f10dd9e07da2<br>
> Ancestors: Monticello-cmm.726<br>
> <br>
> MCHttpRepository changes:<br>
> - before up- or downloading files, transform the urls using #rewriteUrl:forDownload:. The default rules (see #urlRewriteRules) switch from http to https for source.squeak.org and squeaksource.com, and switch to the the static<br>
> smalltalkhub site for downloads. The url rewriting is Eliot's idea, but this implementation uses a list of rewrite rules instead of a dictionary-based mapping.<br>
> - use WebClient (and the shared webclient instance) for uploads too<br>
> - retry down/uploading with WebClient at most 3 times. This should work around the case where the underlying socket was closed but the state of the socket has not been updated in Squeak.<br>
> - use https in #creationTemplate<br>
> <br>
> =============== Diff against Monticello-cmm.726 ===============<br>
> <br>
> Item was changed:<br>
> MCFileBasedRepository subclass: #MCHttpRepository<br>
> instanceVariableNames: 'location user password readerCache indexed webClient'<br>
> + classVariableNames: 'URLRewriteRules UseSharedWebClientInstance'<br>
> - classVariableNames: 'UseSharedWebClientInstance'<br>
> poolDictionaries: ''<br>
> category: 'Monticello-Repositories'!<br>
> <br>
> Item was changed:<br>
> ----- Method: MCHttpRepository class>>creationTemplate (in category 'ui-support') -----<br>
> creationTemplate<br>
> + ^self creationTemplateLocation: '<a href="https://www.squeaksource.com/ProjectName">https://www.squeaksource.com/ProjectName</a>'<br>
> - ^self creationTemplateLocation: '<a href="http://www.squeaksource.com/ProjectName">http://www.squeaksource.com/ProjectName</a>'<br>
> user: 'squeak'<br>
> password: 'squeak'<br>
> !<br>
> <br>
> Item was added:<br>
> + ----- Method: MCHttpRepository class>>rewriteUrl:forDownload: (in category 'url rewrite') -----<br>
> + rewriteUrl: aString forDownload: forDownload<br>
> +<br>
> + | result |<br>
> + result := aString.<br>
> + self urlRewriteRules groupsDo: [ :regexString :replacement :downloadOnly |<br>
> + (forDownload or: [ downloadOnly not ]) ifTrue: [<br>
> + result := result copyWithRegex: regexString matchesReplacedWith: replacement ] ].<br>
> + ^result<br>
> + <br>
> + "<br>
> + self assert: '<a href="https://squeaksource.com/foo/bar?baz=1">https://squeaksource.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://squeaksource.com/foo/bar?baz=1">http://squeaksource.com/foo/bar?baz=1</a>' forDownload: true).<br>
> + self assert: '<a href="https://squeaksource.com/foo/bar?baz=1">https://squeaksource.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="https://squeaksource.com/foo/bar?baz=1">https://squeaksource.com/foo/bar?baz=1</a>' forDownload: true).<br>
> + self assert: '<a href="https://source.squeak.org/foo/bar?baz=1">https://source.squeak.org/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://source.squeak.org/foo/bar?baz=1">http://source.squeak.org/foo/bar?baz=1</a>' forDownload: true).<br>
> + self assert: '<a href="https://source.squeak.org/foo/bar?baz=1">https://source.squeak.org/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="https://source.squeak.org/foo/bar?baz=1">https://source.squeak.org/foo/bar?baz=1</a>' forDownload: true).<br>
> + self assert: '<a href="http://static.smalltalkhub.com/foo/bar?baz=1">http://static.smalltalkhub.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://smalltalkhub.com/foo/bar?baz=1">http://smalltalkhub.com/foo/bar?baz=1</a>' forDownload: true).<br>
> + self assert: '<a href="http://smalltalkhub.com/foo/bar?baz=1">http://smalltalkhub.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://smalltalkhub.com/foo/bar?baz=1">http://smalltalkhub.com/foo/bar?baz=1</a>' forDownload: false).<br>
> + "!<br>
> <br>
> Item was added:<br>
> + ----- Method: MCHttpRepository class>>urlRewriteRules (in category 'url rewrite') -----<br>
> + urlRewriteRules<br>
> +<br>
> + ^URLRewriteRules ifNil: [<br>
> + URLRewriteRules := #(<br>
> + "Regex to be replaced" "static replacement string" "download only"<br>
> + '^http\://source\.squeak\.org/' '<a href="https://source.squeak.org/">https://source.squeak.org/</a>' false<br>
> + '^http\://squeaksource\.com/' '<a href="https://squeaksource.com/">https://squeaksource.com/</a>' false<br>
> + '^http\://www.squeaksource\.com/' '<a href="https://www.squeaksource.com/">https://www.squeaksource.com/</a>' false<br>
> + '^http\://smalltalkhub.com/' '<a href="http://static.smalltalkhub.com/">http://static.smalltalkhub.com/</a>' true <br>
> + ) asOrderedCollection ]!<br>
> <br>
> Item was changed:<br>
> ----- Method: MCHttpRepository>>httpGet:arguments: (in category 'private') -----<br>
> httpGet: url arguments: arguments<br>
> <br>
> + | urlString |<br>
> - | progress urlString client response result |<br>
> - progress := [ :total :amount |<br>
> - HTTPProgress new<br>
> - total: total;<br>
> - amount: amount;<br>
> - signal: 'Downloading...' ].<br>
> urlString := arguments<br>
> ifNil: [ url ]<br>
> ifNotNil: [<br>
> | queryString |<br>
> queryString := WebUtils encodeUrlEncodedForm: arguments.<br>
> (url includes: $?)<br>
> ifTrue: [ url, '&', queryString ]<br>
> ifFalse: [ url, '?', queryString ] ].<br>
> + urlString := self class rewriteUrl: urlString forDownload: true.<br>
> + ^self webClientDo: [ :client |<br>
> + client<br>
> + username: self user;<br>
> + password: self password;<br>
> + httpGet: urlString do: [ :request |<br>
> + request<br>
> + headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;<br>
> + headerAt: 'Connection' put: 'Keep-Alive';<br>
> + headerAt: 'Accept' put: '*/*' ] ]!<br>
> - self class useSharedWebClientInstance ifTrue: [<br>
> - "Acquire webClient by atomically storing it in the client variable and setting its value to nil."<br>
> - client := webClient.<br>
> - webClient := nil ].<br>
> - client<br>
> - ifNil: [ client := WebClient new ]<br>
> - ifNotNil: [<br>
> - "Attempt to avoid an error on windows by recreating the underlying stream."<br>
> - client isConnected ifFalse: [ client close ] ].<br>
> - response := client<br>
> - username: self user;<br>
> - password: self password;<br>
> - httpGet: urlString do: [ :request |<br>
> - request<br>
> - headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;<br>
> - headerAt: 'Connection' put: 'Keep-Alive';<br>
> - headerAt: 'Accept' put: '*/*' ].<br>
> - result := (response code between: 200 and: 299)<br>
> - ifFalse: [<br>
> - response content. "Make sure content is read."<br>
> - nil ]<br>
> - ifTrue: [ (RWBinaryOrTextStream with: (response contentWithProgress: progress)) reset ].<br>
> - self class useSharedWebClientInstance<br>
> - ifTrue: [<br>
> - "Save the WebClient instance for reuse, but only if there is no client cached."<br>
> - webClient <br>
> - ifNil: [ webClient := client ]<br>
> - ifNotNil: [ client close ] ]<br>
> - ifFalse: [ client close ].<br>
> - result ifNil: [ NetworkError signal: 'Could not access ', location ].<br>
> - ^result!<br>
> <br>
> Item was added:<br>
> + ----- Method: MCHttpRepository>>webClientDo: (in category 'private') -----<br>
> + webClientDo: aBlock<br>
> +<br>
> + | client attemptsLeft response result |<br>
> + self class useSharedWebClientInstance ifTrue: [<br>
> + "Acquire webClient by atomically storing it in the client variable and setting its value to nil."<br>
> + client := webClient.<br>
> + webClient := nil ].<br>
> + <br>
> + client<br>
> + ifNil: [ client := WebClient new ]<br>
> + ifNotNil: [<br>
> + "Attempt to avoid an error by recreating the underlying stream."<br>
> + client isConnected ifFalse: [ client close ] ].<br>
> + <br>
> + attemptsLeft := 3.<br>
> + response := nil.<br>
> + [ response isNil and: [ attemptsLeft > 0 ] ] whileTrue: [<br>
> + response := [ aBlock value: client ]<br>
> + on: NetworkError<br>
> + do: [ :error |<br>
> + attemptsLeft = 0 ifTrue: [ error pass ].<br>
> + (3 - attemptsLeft) seconds asDelay wait.<br>
> + attemptsLeft := attemptsLeft - 1.<br>
> + nil "The response" ] ].<br>
> + <br>
> + result := (response code between: 200 and: 299)<br>
> + ifFalse: [<br>
> + response content. "Make sure content is read."<br>
> + nil ]<br>
> + ifTrue: [<br>
> + (RWBinaryOrTextStream with: (<br>
> + response contentWithProgress: [ :total :amount |<br>
> + HTTPProgress new<br>
> + total: total;<br>
> + amount: amount;<br>
> + signal ])) reset ].<br>
> +<br>
> + self class useSharedWebClientInstance<br>
> + ifTrue: [<br>
> + "Save the WebClient instance for reuse, but only if there is no client cached."<br>
> + webClient <br>
> + ifNil: [ webClient := client ]<br>
> + ifNotNil: [ client close ] ]<br>
> + ifFalse: [ client close ].<br>
> +<br>
> + result ifNil: [ NetworkError signal: 'Could not access ', location ].<br>
> + ^result!<br>
> <br>
> Item was changed:<br>
> ----- Method: MCHttpRepository>>writeStreamForFileNamed:replace:do: (in category 'private') -----<br>
> writeStreamForFileNamed: aString replace: ignoreBoolean do: aBlock<br>
> +<br>
> + | stream urlString |<br>
> - | stream response statusLine code |<br>
> stream := RWBinaryOrTextStream on: String new.<br>
> aBlock value: stream.<br>
> + urlString := self urlForFileNamed: aString.<br>
> + urlString := self class rewriteUrl: urlString forDownload: false.<br>
> + ^self displayProgress: 'Uploading ', aString during: [<br>
> + self webClientDo: [ :client |<br>
> + client<br>
> + username: self user;<br>
> + password: self password;<br>
> + httpPut: urlString<br>
> + content: stream contents<br>
> + type: nil<br>
> + do: [ :request |<br>
> + request<br>
> + headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;<br>
> + headerAt: 'Connection' put: 'Keep-Alive';<br>
> + headerAt: 'Accept' put: '*/*' ] ] ]!<br>
> - self displayProgress: 'Uploading ', aString during:[<br>
> - response := HTTPSocket<br>
> - httpPut: stream contents<br>
> - to: (self urlForFileNamed: aString)<br>
> - user: self user<br>
> - passwd: self password.<br>
> - ].<br>
> - "More robust handling of HTTP responses. Instead of enumerating<br>
> - all possible return codes and http versions, do a quick parse"<br>
> - (response beginsWith: 'HTTP/') ifTrue:[<br>
> - "Looks like an HTTP header, not some error message"<br>
> - statusLine := response copyUpTo: Character cr.<br>
> - code := [(statusLine findTokens: ' ') second asInteger] on: Error do:[].<br>
> - ].<br>
> - (code isInteger and:[code between: 200 and: 299])<br>
> - ifFalse:[self error: response].!<br>
> <br>
> <br>
> <br>
><br>
</div>
</span></font>
</body>
</html>