<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<meta content="text/html; charset=UTF-8">
<style type="text/css" style="">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif">
<p>Hi Levente,</p>
<p><br>
</p>
<p>thanks for the review! Yeah, you're right, there must be some (maybe historic) bug when parsing lookaround expressions. I will have a look at this as soon as possible.</p>
<p>However, I'm quite busy at the moment, so please forgive me if I won't be able to fix this until next week ...</p>
<p><br>
</p>
<p>> <span style="font-size:12pt">For bonus points, it would be great to support non-capturing groups too</span></p>
<div>> (?:). :)</div>
<div><br>
</div>
<div>Yeah, this is a great idea! I already had implemented this in my image some time ago, but I did not yet commit it to the inbox (tests are not yet complete).</div>
<div>Regex is a really exciting domain. I'm also having named captures in my pipeline. I'm looking forward to completing work on these features! :-)</div>
<div><span><br>
</span></div>
<div><span>Best,</span></div>
<div><span>Christoph</span></div>
<p></p>
<div id="x_Signature">
<div id="x_divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<div name="x_divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div><font size="2" color="#808080"></font></div>
</div>
</div>
</div>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>Von:</b> Squeak-dev <squeak-dev-bounces@lists.squeakfoundation.org> im Auftrag von Levente Uzonyi <leves@caesar.elte.hu><br>
<b>Gesendet:</b> Mittwoch, 15. April 2020 10:30:49<br>
<b>An:</b> squeak-dev@lists.squeakfoundation.org<br>
<b>Betreff:</b> Re: [squeak-dev] The Inbox: Regex-Core-ct.56.mcz</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi Christoph,<br>
<br>
I finally had a look at these changes. I didn't look at the <br>
implementation in depth, but I found two issues:<br>
<br>
1) the lookarounds behave as capturing groups. They even introduce <br>
spurious subexpressions. Here's an example:<br>
<br>
'(?<=a)b' asRegex search: 'ab'; in: [ :regex |<br>
(1 to: regex subexpressionCount)<br>
collect: [ :index | regex subexpressions: index ]<br>
as: Array ].<br>
<br>
which gives #(#('b') #('') #('a')) instead of just #(#('b')).<br>
<br>
A regex with a regular capturing group<br>
<br>
'(a)b' asRegex search: 'ab'; in: [ :regex |<br>
(1 to: regex subexpressionCount)<br>
collect: [ :index | regex subexpressions: index ]<br>
as: Array ]<br>
<br>
gives the expected #(#('ab') #('a')).<br>
<br>
2) lookarounds can be quantified, and when they are, they behave <br>
unexpectedly. For example, the following raises an error during #search:<br>
<br>
'(?<=a)?b' asRegex search: 'ab'<br>
<br>
IMO quantification doesn't make sense here. There should be an error <br>
raised by #asRegex when a lookaround group is quantified. If anything <br>
needs to be quantified, it can be done inside the lookaround group.<br>
<br>
When the lookarounds are used as expected, they seem to work as expected.<br>
But I think these issues need to be addressed before these changes are <br>
integrated.<br>
For bonus points, it would be great to support non-capturing groups too <br>
(?:). :)<br>
<br>
<br>
Levente<br>
<br>
On Fri, 6 Mar 2020, commits@source.squeak.org wrote:<br>
<br>
> Christoph Thiede uploaded a new version of Regex-Core to project The Inbox:<br>
> <a href="http://source.squeak.org/inbox/Regex-Core-ct.56.mcz">http://source.squeak.org/inbox/Regex-Core-ct.56.mcz</a><br>
><br>
> ==================== Summary ====================<br>
><br>
> Name: Regex-Core-ct.56<br>
> Author: ct<br>
> Time: 6 March 2020, 8:32:34.316886 pm<br>
> UUID: 22955049-895d-ca43-8919-1d9f44b851f9<br>
> Ancestors: Regex-Core-ct.55<br>
><br>
> Implements lookbehinds (both positive and negative) in Regular Expressions for Squeak.<br>
><br>
> '(?<=(?<!n''t\s+)l[o]ve\s+)\w+' asRegex matchesIn: 'I love Squeak. I don''t love C++. I love Smalltalk.'.<br>
><br>
> - Honor lookbehind syntax in RxParser (see #lookAround)<br>
> - Add #forward argument to nodes/links messages. Extend RxsLookaround state by #forward boolean.<br>
> - Create RxmLookahead as universal class for both lookahead and lookbehind links. Remove RxmLookahead.<br>
> - Implement actual lookbehind logic in RxMatcher >> #matchAgainstLookbehind:positive:nextLink:.<br>
><br>
> Again, I decided not to give further support for the #forward-less versions of the relevant messages, nor for the class RxmLookahead. First, I'm not sure whether "lookahead" is a universally useful default assumption for "lookaround". Also, these selectors
have been introduced just a few hours ago in Regex-Core-ct.55, so there should not be a high demand for backward compatibility. Second, all these things are hidden behind the RxMatcher facade, so foreign clients should not really depend on them. (Bad pun:
Don't look-behind the facade ...) Opinions on this topic are highly appreciated.<br>
><br>
> This commit depends indeed on Regex-Core-ct.55. Please review carefully! The lookbehind matching implementation uses an intuitive algorithm. I give absolutely no guarantee that this is an efficient implementation, but for the beginning, even a superpolynomial
complexity should be better than no implementation at all, shouldn't it? :-) Further information about lookbehinds can be found here:
<a href="https://www.regular-expressions.info/lookaround.html">https://www.regular-expressions.info/lookaround.html</a><br>
><br>
> =============== Diff against Regex-Core-ct.55 ===============<br>
><br>
> Item was added:<br>
> + ----- Method: RxMatchOptimizer>>syntaxLookaround:forward:positive: (in category 'double dispatch') -----<br>
> + syntaxLookaround: lookaroundNode forward: forward positive: positive<br>
> + "Do nothing."!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxMatchOptimizer>>syntaxLookaround:positive: (in category 'double dispatch') -----<br>
> - syntaxLookaround: lookaroundNode positive: positive<br>
> - "Do nothing."!<br>
><br>
> Item was added:<br>
> + ----- Method: RxMatcher>>matchAgainstLookbehind:positive:nextLink: (in category 'matching') -----<br>
> + matchAgainstLookbehind: lookbehind positive: positive nextLink: anRmxLink<br>
> + <br>
> + | position matchesLookbehind |<br>
> + position := stream position.<br>
> + matchesLookbehind := (position to: 0 by: -1)<br>
> + anySatisfy: [:index |<br>
> + stream position: index.<br>
> + (lookbehind matchAgainst: self)<br>
> + and: [stream position = position]].<br>
> + matchesLookbehind = positive<br>
> + ifFalse: [^ false].<br>
> + stream position: position.<br>
> + ^ anRmxLink matchAgainst: self!<br>
><br>
> Item was added:<br>
> + ----- Method: RxMatcher>>syntaxLookaround:forward:positive: (in category 'double dispatch') -----<br>
> + syntaxLookaround: lookaroundNode forward: forwardBoolean positive: positiveBoolean<br>
> + "Double dispatch from the syntax tree.<br>
> + Special link can handle lookarounds (look ahead and look behind, positive and negative)."<br>
> + | piece |<br>
> + piece := lookaroundNode piece dispatchTo: self.<br>
> + ^ RxmLookaround with: piece forward: forwardBoolean positive: positiveBoolean!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxMatcher>>syntaxLookaround:positive: (in category 'double dispatch') -----<br>
> - syntaxLookaround: lookaroundNode positive: positiveBoolean<br>
> - "Double dispatch from the syntax tree.<br>
> - Special link can handle lookarounds (look ahead, positive and negative)."<br>
> - | piece |<br>
> - piece := lookaroundNode piece dispatchTo: self.<br>
> - ^ RxmLookahead with: piece positive: positiveBoolean!<br>
><br>
> Item was changed:<br>
> ----- Method: RxParser>>lookAround (in category 'recursive descent') -----<br>
> lookAround<br>
> "Parse a lookaround expression after: (?<lookaround>)<br>
> <lookaround> ::= !!<regex> | =<regex>"<br>
> + | lookbehind positive |<br>
> + ('!!=<' includes: lookahead) ifFalse: [<br>
> - | positive |<br>
> - ('!!=' includes: lookahead) ifFalse: [<br>
> ^ self signalParseError: 'Invalid lookaround expression ?', lookahead asString].<br>
> + lookbehind := lookahead == $<<br>
> + ifTrue: [self next];<br>
> + yourself.<br>
> positive := lookahead == $=.<br>
> self next.<br>
> ^ RxsLookaround<br>
> with: self regex<br>
> + forward: lookbehind not<br>
> positive: positive!<br>
><br>
> Item was removed:<br>
> - RxmLink subclass: #RxmLookahead<br>
> - instanceVariableNames: 'lookahead positive'<br>
> - classVariableNames: ''<br>
> - poolDictionaries: ''<br>
> - category: 'Regex-Core'!<br>
> - <br>
> - !RxmLookahead commentStamp: 'ct 3/6/2020 18:29' prior: 0!<br>
> - Instance holds onto a lookahead which matches but does not consume anything.<br>
> - <br>
> - Instance Variables<br>
> - lookahead: <RxmLink><br>
> - positive: <Boolean><br>
> - !<br>
><br>
> Item was removed:<br>
> - ----- Method: RxmLookahead class>>with:positive: (in category 'instance creation') -----<br>
> - with: aPiece positive: aBoolean<br>
> - <br>
> - ^self new lookahead: aPiece positive: aBoolean!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxmLookahead>>lookahead:positive: (in category 'accessing') -----<br>
> - lookahead: anRxmLink positive: aBoolean<br>
> - lookahead := anRxmLink.<br>
> - positive := aBoolean.!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxmLookahead>>matchAgainst: (in category 'matching') -----<br>
> - matchAgainst: aMatcher<br>
> - "Match if the predicate block evaluates to true when given the<br>
> - current stream character as the argument."<br>
> - <br>
> - ^aMatcher matchAgainstLookahead: lookahead positive: positive nextLink: next!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxmLookahead>>postCopy (in category 'copying') -----<br>
> - postCopy<br>
> - <br>
> - super postCopy.<br>
> - lookahead := lookahead copy!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxmLookahead>>postCopyUsing: (in category 'copying') -----<br>
> - postCopyUsing: anIdentityDictionary<br>
> - <br>
> - super postCopyUsing: anIdentityDictionary.<br>
> - lookahead := lookahead copyUsing: anIdentityDictionary!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxmLookahead>>terminateWith: (in category 'building') -----<br>
> - terminateWith: aNode<br>
> - lookahead terminateWith: aNode.<br>
> - super terminateWith: aNode.!<br>
><br>
> Item was added:<br>
> + RxmLink subclass: #RxmLookaround<br>
> + instanceVariableNames: 'forward positive lookaround'<br>
> + classVariableNames: ''<br>
> + poolDictionaries: ''<br>
> + category: 'Regex-Core'!<br>
> + <br>
> + !RxmLookaround commentStamp: 'ct 3/6/2020 19:45' prior: 0!<br>
> + Instance holds onto a lookaround which matches but does not consume anything.<br>
> + <br>
> + Instance Variables<br>
> + lookbehind: <RxmLink><br>
> + forward: <Boolean><br>
> + positive: <Boolean>!<br>
><br>
> Item was added:<br>
> + ----- Method: RxmLookaround class>>with:forward:positive: (in category 'instance creation') -----<br>
> + with: aPiece forward: forwardBoolean positive: positiveBoolean<br>
> + <br>
> + ^self new lookaround: aPiece forward: forwardBoolean positive: positiveBoolean!<br>
><br>
> Item was added:<br>
> + ----- Method: RxmLookaround>>lookaround:forward:positive: (in category 'accessing') -----<br>
> + lookaround: anRxmLink forward: forwardBoolean positive: positiveBoolean<br>
> + lookaround := anRxmLink.<br>
> + forward := forwardBoolean.<br>
> + positive := positiveBoolean.!<br>
><br>
> Item was added:<br>
> + ----- Method: RxmLookaround>>matchAgainst: (in category 'matching') -----<br>
> + matchAgainst: aMatcher<br>
> + "Match if the predicate block evaluates to true when given the current stream character as the argument."<br>
> + <br>
> + ^ forward<br>
> + ifTrue: [aMatcher matchAgainstLookahead: lookaround positive: positive nextLink: next]<br>
> + ifFalse: [aMatcher matchAgainstLookbehind: lookaround positive: positive nextLink: next]!<br>
><br>
> Item was added:<br>
> + ----- Method: RxmLookaround>>postCopy (in category 'copying') -----<br>
> + postCopy<br>
> + <br>
> + super postCopy.<br>
> + lookaround := lookaround copy!<br>
><br>
> Item was added:<br>
> + ----- Method: RxmLookaround>>postCopyUsing: (in category 'copying') -----<br>
> + postCopyUsing: anIdentityDictionary<br>
> + <br>
> + super postCopyUsing: anIdentityDictionary.<br>
> + lookaround := lookaround copyUsing: anIdentityDictionary!<br>
><br>
> Item was added:<br>
> + ----- Method: RxmLookaround>>terminateWith: (in category 'building') -----<br>
> + terminateWith: aNode<br>
> + lookaround terminateWith: aNode.<br>
> + super terminateWith: aNode.!<br>
><br>
> Item was changed:<br>
> RxsNode subclass: #RxsLookaround<br>
> + instanceVariableNames: 'piece forward positive'<br>
> - instanceVariableNames: 'piece positive'<br>
> classVariableNames: ''<br>
> poolDictionaries: ''<br>
> category: 'Regex-Core'!<br>
> <br>
> + !RxsLookaround commentStamp: 'ct 3/6/2020 18:31' prior: 0!<br>
> + Lookaround is used for lookaheads and lookbehinds. They are used to check if the input matches a certain subexpression without consuming any characters (e.g. not advancing the match position).<br>
> - !RxsLookaround commentStamp: '<historical>' prior: 0!<br>
> - I lookaround is used for lookaheads and lookbehinds. They are used to check if the input matches a certain subexpression without consuming any characters (e.g. not advancing the match position).<br>
><br>
> Lookarounds can be positive or negative. If they are positive the condition fails if the subexpression fails, if they are negative it is inverse.!<br>
><br>
> Item was added:<br>
> + ----- Method: RxsLookaround class>>with:forward:positive: (in category 'instance creation') -----<br>
> + with: aRxsRegex forward: forwardBoolean positive: positiveBoolean<br>
> + ^ self new<br>
> + initializePiece: aRxsRegex<br>
> + forward: forwardBoolean<br>
> + positive: positiveBoolean!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxsLookaround class>>with:positive: (in category 'instance creation') -----<br>
> - with: aRxsRegex positive: positiveBoolean<br>
> - ^ self new<br>
> - initializePiece: aRxsRegex<br>
> - positive: positiveBoolean!<br>
><br>
> Item was added:<br>
> + ----- Method: RxsLookaround>>beLookahead (in category 'initialize-release') -----<br>
> + beLookahead<br>
> + forward := true!<br>
><br>
> Item was added:<br>
> + ----- Method: RxsLookaround>>beLookbehind (in category 'initialize-release') -----<br>
> + beLookbehind<br>
> + forward := false!<br>
><br>
> Item was changed:<br>
> ----- Method: RxsLookaround>>dispatchTo: (in category 'accessing') -----<br>
> dispatchTo: aBuilder<br>
> "Inform the matcher of the kind of the node, and it will do whatever it has to."<br>
> + ^aBuilder syntaxLookaround: self forward: self forward positive: self positive!<br>
> - ^aBuilder syntaxLookaround: self positive: self positive!<br>
><br>
> Item was added:<br>
> + ----- Method: RxsLookaround>>forward (in category 'accessing') -----<br>
> + forward<br>
> + <br>
> + ^ forward!<br>
><br>
> Item was added:<br>
> + ----- Method: RxsLookaround>>initializePiece:forward:positive: (in category 'initialize-release') -----<br>
> + initializePiece: anRsxPiece forward: forwardBoolean positive: positiveBoolean<br>
> + <br>
> + piece := anRsxPiece.<br>
> + forward := forwardBoolean.<br>
> + positive := positiveBoolean.!<br>
><br>
> Item was removed:<br>
> - ----- Method: RxsLookaround>>initializePiece:positive: (in category 'initialize-release') -----<br>
> - initializePiece: anRsxPiece positive: positiveBoolean<br>
> - <br>
> - piece := anRsxPiece.<br>
> - positive := positiveBoolean.!<br>
<br>
</div>
</span></font>
</body>
</html>