<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Very nice idea! :-) If I wanted to hijack an image, these <span>URLRewriteRules would probably be my first approach - but security has never been an issue for Squeak, so I guess this is not a problem.</span></p>
<p><span><br>
</span></p>
<p><span>Does this also fix the problem with the classic HTTP URLs returned by the update map or will we still need to patch them on the server side?</span></p>
<p><span><br>
</span></p>
<p><span>And one last question regarding to your tests in the method comment of #<span>rewriteUrl:forDownload:: Couldn't you put them into a real test case? I'm pretty sure that not everyone will run these out-commented tests manually, and it would be a pity
not to automate them.</span></span></p>
<p><span><span><br>
</span></span></p>
<p><span><span>Best,</span></span></p>
<p><span><span>Christoph</span></span></p>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div name="x_divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div><font size="2" color="#808080"></font></div>
</div>
</div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Von:</b> Squeak-dev <squeak-dev-bounces@lists.squeakfoundation.org> im Auftrag von commits@source.squeak.org <commits@source.squeak.org><br>
<b>Gesendet:</b> Donnerstag, 17. September 2020 01:55:41<br>
<b>An:</b> squeak-dev@lists.squeakfoundation.org<br>
<b>Betreff:</b> [squeak-dev] The Inbox: Monticello-ul.727.mcz</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Levente Uzonyi uploaded a new version of Monticello to project The Inbox:<br>
<a href="http://source.squeak.org/inbox/Monticello-ul.727.mcz">http://source.squeak.org/inbox/Monticello-ul.727.mcz</a><br>
<br>
==================== Summary ====================<br>
<br>
Name: Monticello-ul.727<br>
Author: ul<br>
Time: 17 September 2020, 1:54:51.056164 am<br>
UUID: ad776836-42eb-4aa2-b788-f10dd9e07da2<br>
Ancestors: Monticello-cmm.726<br>
<br>
MCHttpRepository changes:<br>
- before up- or downloading files, transform the urls using #rewriteUrl:forDownload:. The default rules (see #urlRewriteRules) switch from http to https for source.squeak.org and squeaksource.com, and switch to the the static smalltalkhub site for downloads.
The url rewriting is Eliot's idea, but this implementation uses a list of rewrite rules instead of a dictionary-based mapping.<br>
- use WebClient (and the shared webclient instance) for uploads too<br>
- retry down/uploading with WebClient at most 3 times. This should work around the case where the underlying socket was closed but the state of the socket has not been updated in Squeak.<br>
- use https in #creationTemplate<br>
<br>
=============== Diff against Monticello-cmm.726 ===============<br>
<br>
Item was changed:<br>
MCFileBasedRepository subclass: #MCHttpRepository<br>
instanceVariableNames: 'location user password readerCache indexed webClient'<br>
+ classVariableNames: 'URLRewriteRules UseSharedWebClientInstance'<br>
- classVariableNames: 'UseSharedWebClientInstance'<br>
poolDictionaries: ''<br>
category: 'Monticello-Repositories'!<br>
<br>
Item was changed:<br>
----- Method: MCHttpRepository class>>creationTemplate (in category 'ui-support') -----<br>
creationTemplate<br>
+ ^self creationTemplateLocation: '<a href="https://www.squeaksource.com/ProjectName">https://www.squeaksource.com/ProjectName</a>'<br>
- ^self creationTemplateLocation: '<a href="http://www.squeaksource.com/ProjectName">http://www.squeaksource.com/ProjectName</a>'<br>
user: 'squeak'<br>
password: 'squeak'<br>
!<br>
<br>
Item was added:<br>
+ ----- Method: MCHttpRepository class>>rewriteUrl:forDownload: (in category 'url rewrite') -----<br>
+ rewriteUrl: aString forDownload: forDownload<br>
+ <br>
+ | result |<br>
+ result := aString.<br>
+ self urlRewriteRules groupsDo: [ :regexString :replacement :downloadOnly |<br>
+ (forDownload or: [ downloadOnly not ]) ifTrue: [<br>
+ result := result copyWithRegex: regexString matchesReplacedWith: replacement ] ].<br>
+ ^result<br>
+ <br>
+ "<br>
+ self assert: '<a href="https://squeaksource.com/foo/bar?baz=1">https://squeaksource.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://squeaksource.com/foo/bar?baz=1">http://squeaksource.com/foo/bar?baz=1</a>' forDownload: true).<br>
+ self assert: '<a href="https://squeaksource.com/foo/bar?baz=1">https://squeaksource.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="https://squeaksource.com/foo/bar?baz=1">https://squeaksource.com/foo/bar?baz=1</a>' forDownload: true).<br>
+ self assert: '<a href="https://source.squeak.org/foo/bar?baz=1">https://source.squeak.org/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://source.squeak.org/foo/bar?baz=1">http://source.squeak.org/foo/bar?baz=1</a>' forDownload: true).<br>
+ self assert: '<a href="https://source.squeak.org/foo/bar?baz=1">https://source.squeak.org/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="https://source.squeak.org/foo/bar?baz=1">https://source.squeak.org/foo/bar?baz=1</a>' forDownload: true).<br>
+ self assert: '<a href="http://static.smalltalkhub.com/foo/bar?baz=1">http://static.smalltalkhub.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://smalltalkhub.com/foo/bar?baz=1">http://smalltalkhub.com/foo/bar?baz=1</a>' forDownload: true).<br>
+ self assert: '<a href="http://smalltalkhub.com/foo/bar?baz=1">http://smalltalkhub.com/foo/bar?baz=1</a>' = (self rewriteUrl: '<a href="http://smalltalkhub.com/foo/bar?baz=1">http://smalltalkhub.com/foo/bar?baz=1</a>' forDownload: false).<br>
+ "!<br>
<br>
Item was added:<br>
+ ----- Method: MCHttpRepository class>>urlRewriteRules (in category 'url rewrite') -----<br>
+ urlRewriteRules<br>
+ <br>
+ ^URLRewriteRules ifNil: [<br>
+ URLRewriteRules := #(<br>
+ "Regex to be replaced" "static replacement string" "download only"<br>
+ '^http\://source\.squeak\.org/' '<a href="https://source.squeak.org/">https://source.squeak.org/</a>' false<br>
+ '^http\://squeaksource\.com/' '<a href="https://squeaksource.com/">https://squeaksource.com/</a>' false<br>
+ '^http\://www.squeaksource\.com/' '<a href="https://www.squeaksource.com/">https://www.squeaksource.com/</a>' false<br>
+ '^http\://smalltalkhub.com/' '<a href="http://static.smalltalkhub.com/">http://static.smalltalkhub.com/</a>' true
<br>
+ ) asOrderedCollection ]!<br>
<br>
Item was changed:<br>
----- Method: MCHttpRepository>>httpGet:arguments: (in category 'private') -----<br>
httpGet: url arguments: arguments<br>
<br>
+ | urlString |<br>
- | progress urlString client response result |<br>
- progress := [ :total :amount |<br>
- HTTPProgress new <br>
- total: total;<br>
- amount: amount;<br>
- signal: 'Downloading...' ].<br>
urlString := arguments<br>
ifNil: [ url ]<br>
ifNotNil: [ <br>
| queryString |<br>
queryString := WebUtils encodeUrlEncodedForm: arguments.<br>
(url includes: $?)<br>
ifTrue: [ url, '&', queryString ]<br>
ifFalse: [ url, '?', queryString ] ].<br>
+ urlString := self class rewriteUrl: urlString forDownload: true.<br>
+ ^self webClientDo: [ :client | <br>
+ client<br>
+ username: self user;<br>
+ password: self password;<br>
+ httpGet: urlString do: [ :request |<br>
+ request<br>
+ headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;<br>
+ headerAt: 'Connection' put: 'Keep-Alive';<br>
+ headerAt: 'Accept' put: '*/*' ] ]!<br>
- self class useSharedWebClientInstance ifTrue: [<br>
- "Acquire webClient by atomically storing it in the client variable and setting its value to nil."<br>
- client := webClient.<br>
- webClient := nil ].<br>
- client <br>
- ifNil: [ client := WebClient new ]<br>
- ifNotNil: [ <br>
- "Attempt to avoid an error on windows by recreating the underlying stream."<br>
- client isConnected ifFalse: [ client close ] ].<br>
- response := client<br>
- username: self user;<br>
- password: self password;<br>
- httpGet: urlString do: [ :request |<br>
- request<br>
- headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;<br>
- headerAt: 'Connection' put: 'Keep-Alive';<br>
- headerAt: 'Accept' put: '*/*' ].<br>
- result := (response code between: 200 and: 299) <br>
- ifFalse: [<br>
- response content. "Make sure content is read."<br>
- nil ]<br>
- ifTrue: [ (RWBinaryOrTextStream with: (response contentWithProgress: progress)) reset ].<br>
- self class useSharedWebClientInstance<br>
- ifTrue: [<br>
- "Save the WebClient instance for reuse, but only if there is no client cached."<br>
- webClient <br>
- ifNil: [ webClient := client ]<br>
- ifNotNil: [ client close ] ]<br>
- ifFalse: [ client close ].<br>
- result ifNil: [ NetworkError signal: 'Could not access ', location ].<br>
- ^result!<br>
<br>
Item was added:<br>
+ ----- Method: MCHttpRepository>>webClientDo: (in category 'private') -----<br>
+ webClientDo: aBlock<br>
+ <br>
+ | client attemptsLeft response result |<br>
+ self class useSharedWebClientInstance ifTrue: [<br>
+ "Acquire webClient by atomically storing it in the client variable and setting its value to nil."<br>
+ client := webClient.<br>
+ webClient := nil ].<br>
+ <br>
+ client <br>
+ ifNil: [ client := WebClient new ]<br>
+ ifNotNil: [ <br>
+ "Attempt to avoid an error by recreating the underlying stream."<br>
+ client isConnected ifFalse: [ client close ] ].<br>
+ <br>
+ attemptsLeft := 3.<br>
+ response := nil.<br>
+ [ response isNil and: [ attemptsLeft > 0 ] ] whileTrue: [<br>
+ response := [ aBlock value: client ]<br>
+ on: NetworkError<br>
+ do: [ :error |<br>
+ attemptsLeft = 0 ifTrue: [ error pass ].<br>
+ (3 - attemptsLeft) seconds asDelay wait.<br>
+ attemptsLeft := attemptsLeft - 1.<br>
+ nil "The response" ] ]. <br>
+ <br>
+ result := (response code between: 200 and: 299) <br>
+ ifFalse: [<br>
+ response content. "Make sure content is read."<br>
+ nil ]<br>
+ ifTrue: [ <br>
+ (RWBinaryOrTextStream with: (<br>
+ response contentWithProgress: [ :total :amount |<br>
+ HTTPProgress new <br>
+ total: total;<br>
+ amount: amount;<br>
+ signal ])) reset ].<br>
+ <br>
+ self class useSharedWebClientInstance<br>
+ ifTrue: [<br>
+ "Save the WebClient instance for reuse, but only if there is no client cached."<br>
+ webClient <br>
+ ifNil: [ webClient := client ]<br>
+ ifNotNil: [ client close ] ]<br>
+ ifFalse: [ client close ].<br>
+ <br>
+ result ifNil: [ NetworkError signal: 'Could not access ', location ].<br>
+ ^result!<br>
<br>
Item was changed:<br>
----- Method: MCHttpRepository>>writeStreamForFileNamed:replace:do: (in category 'private') -----<br>
writeStreamForFileNamed: aString replace: ignoreBoolean do: aBlock<br>
+ <br>
+ | stream urlString |<br>
- | stream response statusLine code |<br>
stream := RWBinaryOrTextStream on: String new.<br>
aBlock value: stream.<br>
+ urlString := self urlForFileNamed: aString.<br>
+ urlString := self class rewriteUrl: urlString forDownload: false.<br>
+ ^self displayProgress: 'Uploading ', aString during: [<br>
+ self webClientDo: [ :client |<br>
+ client<br>
+ username: self user;<br>
+ password: self password;<br>
+ httpPut: urlString<br>
+ content: stream contents<br>
+ type: nil<br>
+ do: [ :request |<br>
+ request<br>
+ headerAt: 'Authorization' put: 'Basic ', (self user, ':', self password) base64Encoded;<br>
+ headerAt: 'Connection' put: 'Keep-Alive';<br>
+ headerAt: 'Accept' put: '*/*' ] ] ]!<br>
- self displayProgress: 'Uploading ', aString during:[<br>
- response := HTTPSocket<br>
- httpPut: stream contents<br>
- to: (self urlForFileNamed: aString)<br>
- user: self user<br>
- passwd: self password.<br>
- ].<br>
- "More robust handling of HTTP responses. Instead of enumerating<br>
- all possible return codes and http versions, do a quick parse"<br>
- (response beginsWith: 'HTTP/') ifTrue:[<br>
- "Looks like an HTTP header, not some error message"<br>
- statusLine := response copyUpTo: Character cr.<br>
- code := [(statusLine findTokens: ' ') second asInteger] on: Error do:[].<br>
- ].<br>
- (code isInteger and:[code between: 200 and: 299]) <br>
- ifFalse:[self error: response].!<br>
<br>
<br>
</div>
</span></font>
</body>
</html>