Hi,

I'm currently trying to understand Slang's type inference, and to do so trying to write tests.

I'm using CCodeGenerator >> #compileToTMethodSelector:in: to get the TMethod, but i cannot seem to find how to use the type inference on this TMethod (or in the instance of CCodeGenerator).

Also, I have been wondering what is the semantic of sqInt. I found out it was an alias for a long, but i couldn't find better than that. Are there offsets to find objects in the object memory? Are there cast as pointers?

Thank you in advance :)

Pierre.

On 16/12/2019 08:27, Tobias Pape wrote:

(sorry, short on Time atm.)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenSmalltalk/opensmalltalk-vm/pull/450?email_source=notifications&email_token=AIJPEW7MIZ52B4XAWLM3UPDQY4UXFA5CNFSM4JYBTAVKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEG5YPYI#issuecomment-565938145, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AIJPEW4O6CMJIZPIRGJFE3TQY4UXFANCNFSM4JYBTAVA.

On 16/12/2019 15:02, Guillermo Polito wrote:

Coucou,

El 16 dic 2019, a las 13:07, pierre misse <pierre_misse25@msn.commailto:pierre_misse25@msn.com> escribió:

I'm currently trying to understand Slang's type inference, and to do so trying to write tests.

I'm using CCodeGenerator >> #compileToTMethodSelector:in: to get the TMethod, but i cannot seem to find how to use the type inference on this TMethod (or in the instance of CCodeGenerator).

Did you look at the methods

- #returnType[:] - #typeFor:in:

and their senders?

There is also a method protocol “type inference” with methods that may seem interesting. Indeed, but didn't find something that would allow me to launch type inference on something.

Check for senders to try to understand the callgraph backwards :) I did, got stuck several times, hence me asking.

Also, I have been wondering what is the semantic of sqInt.

sqInt was introduced, if I’m not mistaken, to abstract the vm for the underlying exact type used. This was for the initial effort of compiling the VM both for 32 and 64 bits (thus redefining sqInt for the correct one depending on the case).

I found out it was an alias for a long, but i couldn't find better than that.

Which in linux and osx is 64bit long ;) In Windows a long is 32bits but either there is a conditional define for that case stating a long long, or we are super tied to gcc/clang.

To check the actual definition/size, you can try to do a textual search on the entire vmmaker using “method source with it”.

Are there offsets to find objects in the object memory?

Objects in memory have variable sizes (an array of 10 elements is larger than an array of 2) so there is no such thing as fixed offsets. Instead, each object has its own size encoded in its header (hidden from the image). The VM uses that size to find where the object finishes, and where the next object starts. I know that part, but I was thinking that an OOP might be an offsets from the start of the object memory, which would point to the start of the object.

You may check:

- #objectAfter: - https://clementbera.wordpress.com/2014/01/16/spurs-new-object-format/

Are there cast as pointers?

Well, technically sqInts are used as pointers.

Check #longAt: and senders.

Thank you in advance :)

Pierre.

Pierre

Hi Eliot,

Thanks for the detailed answer and all the pointers !

I must have misunderstood #inferTypesForImplicitlyTypedVariablesAndMethods when I first came across it.

Have a nice day :)

Pierre

On 17/12/2019 01:14, Eliot Miranda wrote:

Hi Pierre,

On Mon, Dec 16, 2019 at 4:07 AM pierre misse <pierre_misse25@msn.commailto:pierre_misse25@msn.com> wrote:

Hi,

I'm currently trying to understand Slang's type inference, and to do so trying to write tests.

Is it your intention to contribute those tests back to VMMaker.oscog on source.squeak.orghttp://source.squeak.org? If I am able to, it is my intention.

I'm using CCodeGenerator >> #compileToTMethodSelector:in: to get the TMethod, but i cannot seem to find how to use the type inference on this TMethod (or in the instance of CCodeGenerator).

Here's some doit that run Slang including type inference:

https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/Slang%20Tes...

If you use https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/buildspurtr... to build a VMMaker image it'll create an image containing g this and other useful workspaces that contain doit to launch a VM under the simulator, etc.

Type inference is driven by this loop in CCodeGeneraror>>inferTypesForImplicitlyTypedVariablesAndMethods which iterates, inferring types until no more types can b e inferred (until the algorithm has reached a fixed point).

"now iterate until we reach a fixed point" [| changedReturnType | changedReturnType := false. allMethods do: [:m| m inferTypesForImplicitlyTypedVariablesIn: self. (m inferReturnTypeIn: self) ifTrue: [changedReturnType := true]]. changedReturnType] whileTrue.

Type information is derived from explicit return types in methods (the <returnTypeC: > pragma), types for "kernel" sends from CCodeGenerator>>#returnTypeForSend:in:ifNil:, and for types propagated to returns from variables that themselves are either explicitly typed using <var: ... type: ...> and <var: ... declareC: ...> pragmas, or get their types in assignments of the results of methods, where the type they acquire is the return types of those methods.

Also, I have been wondering what is the semantic of sqInt. I found out it was an alias for a long, but i couldn't find better than that.

#sqInt is first of all a pun for "squint" nice , I was wondering if that was the case too :D , a shortening of Squeak Integer. It is an integer type of the same size as an "ordinary object pointer", or "oop". #usqInt, #sqLong & #usqLong are other important types. #sqLong & #usqLong are 64-bit types. #sqInt & #usqInt are 32-bit types in 32-bit VMs and 64-bit types in 64-bit VMs.

Are there offsets to find objects in the object memory?

A #sqInt is typically used to hold the oop for an object and indeed it points to the header word of the object. The object is preceded by an overflow header word if it has more than 254 slots.

Are there cast as pointers?

Yes. See the functions and macros in platforms/Cross/vm/sqMemoryAccess.h.

Thank you in advance :)

Pierre.

_,,,^..^,,,_ best, Eliot

My first approach is very simple

create a CCodeGenerator add only my current class with independent methods infer the type check the types.

testConstantInt32 | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: self class. tMethod := ccg compileToTMethodSelector: #anInt32 in: self class. ccg inferTypesForImplicitlyTypedVariablesAndMethods. ccg typeFor: tMethod statements first expression variable in: tMethod. "#sqInt" tMethod inferReturnTypeFromReturnsIn: ccg."a TMethod (SlangTypeInferenceTests>>anInt32)" tMethod returnType."nil"

Applied in this case to:

anInt32 | a | ^ a := 30

Maybe I misunderstood the type Inference that I read so far, but based on #TMethod >> #typeFor:in:, I would expect the ccg to keep the type information somewhere, which doesn't seem to be the case.

Maybe it's because my set up is wrong?

This part of the comment puzzles me "deferring to aCodeGen (which defers to the vmClass)", gonna dig into this method.

Thank you,

Pierre

On 17/12/2019 07:33, Eliot Miranda wrote:

Hi Pierre, and welcome.

On Dec 16, 2019, at 10:19 PM, pierre misse pierre_misse25@msn.commailto:pierre_misse25@msn.com wrote: 

Hi Eliot,

Thanks for the detailed answer and all the pointers !

I must have misunderstood #inferTypesForImplicitlyTypedVariablesAndMethods when I first came across it.

Have a nice day :)

thank you :)

Pierre

On 17/12/2019 01:14, Eliot Miranda wrote:

Hi Pierre,

On Mon, Dec 16, 2019 at 4:07 AM pierre misse <pierre_misse25@msn.commailto:pierre_misse25@msn.com> wrote:

Hi,

I'm currently trying to understand Slang's type inference, and to do so trying to write tests.

Is it your intention to contribute those tests back to VMMaker.oscog on source.squeak.orghttp://source.squeak.org? If I am able to, it is my intention.

Thank you. It be warmly appreciated.

One thing yo look at in your tests is instabilities in the algorithm, typically caused by the use of hashed collections (eg Set) that have different enumerations from run to run, usually due to hashes being pseudo-randomly computed. If you look at the last commit of the Spur cointerp.c’s you’ll see one variable that changes type from sqInt to usqInt. This points to such a bug. I’ll try and provide you with specifics soon.

Thanks for the warning. I was actually already aware of this bug, but now I also know the cause !

I'm using CCodeGenerator >> #compileToTMethodSelector:in: to get the TMethod, but i cannot seem to find how to use the type inference on this TMethod (or in the instance of CCodeGenerator).

Here's some doit that run Slang including type inference:

https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/Slang%20Tes...

If you use https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/buildspurtr... to build a VMMaker image it'll create an image containing g this and other useful workspaces that contain doit to launch a VM under the simulator, etc.

Type inference is driven by this loop in CCodeGeneraror>>inferTypesForImplicitlyTypedVariablesAndMethods which iterates, inferring types until no more types can b e inferred (until the algorithm has reached a fixed point).

"now iterate until we reach a fixed point" [| changedReturnType | changedReturnType := false. allMethods do: [:m| m inferTypesForImplicitlyTypedVariablesIn: self. (m inferReturnTypeIn: self) ifTrue: [changedReturnType := true]]. changedReturnType] whileTrue.

Type information is derived from explicit return types in methods (the <returnTypeC: > pragma), types for "kernel" sends from CCodeGenerator>>#returnTypeForSend:in:ifNil:, and for types propagated to returns from variables that themselves are either explicitly typed using <var: ... type: ...> and <var: ... declareC: ...> pragmas, or get their types in assignments of the results of methods, where the type they acquire is the return types of those methods.

Also, I have been wondering what is the semantic of sqInt. I found out it was an alias for a long, but i couldn't find better than that.

#sqInt is first of all a pun for "squint" nice , I was wondering if that was the case too :D , a shortening of Squeak Integer. It is an integer type of the same size as an "ordinary object pointer", or "oop". #usqInt, #sqLong & #usqLong are other important types. #sqLong & #usqLong are 64-bit types. #sqInt & #usqInt are 32-bit types in 32-bit VMs and 64-bit types in 64-bit VMs.

Are there offsets to find objects in the object memory?

A #sqInt is typically used to hold the oop for an object and indeed it points to the header word of the object. The object is preceded by an overflow header word if it has more than 254 slots.

Are there cast as pointers?

Yes. See the functions and macros in platforms/Cross/vm/sqMemoryAccess.h.

Thank you in advance :)

Pierre.

_,,,^..^,,,_ best, Eliot

Cheers

Hi Pierre,

On Tue, Dec 17, 2019 at 9:50 AM pierre misse pierre_misse25@msn.com wrote:

Hi,

I saw my previous mistake, and corrected this. This is a sample test, for a very simple inference.

testReturnFloatConstantNode | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: SlangTypeInferenceTestsClass. ccg inferTypesForImplicitlyTypedVariablesAndMethods. tMethod := ccg methodNamed: #returnAFloatConstantNode.

self assert: tMethod isNotNil.
self assert: (ccg typeFor: tMethod statements first in: tMethod)

equals: #double. self assert: tMethod returnType equals: #double

And perhaps have SlangTypeInferenceTestsClass inherit from StackInterpreter or InterpreterPrimitives. That will give you much more context.

I need to work on ones with ambiguity.

The instability I saw was in getErrorObjectFromPrimFailCode; its two

locals clone & errObj are sometimes typed as squint and sometimes as usqInt. Some times numPointerSlotsOf ends up with a return type of usqInt, some times as sqInt. I've recently had success in another project substituting OrderedDictionary et al for Dictionary. So I'm sure that solutions can be found easily.

Pierre

On 17/12/2019 07:49, pierre misse wrote:

My first approach is very simple

create a CCodeGenerator add only my current class with independent methods infer the type check the types.

testConstantInt32 | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: self class. tMethod := ccg compileToTMethodSelector: #anInt32 in: self class. ccg inferTypesForImplicitlyTypedVariablesAndMethods. ccg typeFor: tMethod statements first expression variable in: tMethod. "#sqInt" tMethod inferReturnTypeFromReturnsIn: ccg."a TMethod (SlangTypeInferenceTests>>anInt32)" tMethod returnType."nil"

Applied in this case to:

anInt32 | a | ^ a := 30

Maybe I misunderstood the type Inference that I read so far, but based on #TMethod >> #typeFor:in:, I would expect the ccg to keep the type information somewhere, which doesn't seem to be the case.

Maybe it's because my set up is wrong?

This part of the comment puzzles me "deferring to aCodeGen (which defers to the vmClass)", gonna dig into this method.

Thank you,

Pierre

On 17/12/2019 07:33, Eliot Miranda wrote:

Hi Pierre, and welcome.

On Dec 16, 2019, at 10:19 PM, pierre misse pierre_misse25@msn.com pierre_misse25@msn.com wrote:



Hi Eliot,

Thanks for the detailed answer and all the pointers !

I must have misunderstood #inferTypesForImplicitlyTypedVariablesAndMethods when I first came across it.

Have a nice day :)

thank you :)

Pierre On 17/12/2019 01:14, Eliot Miranda wrote:

Hi Pierre,

On Mon, Dec 16, 2019 at 4:07 AM pierre misse pierre_misse25@msn.com wrote:

...

Hi,

I'm currently trying to understand Slang's type inference, and to do so trying to write tests.

Is it your intention to contribute those tests back to VMMaker.oscog on source.squeak.org?

If I am able to, it is my intention.

Thank you. It be warmly appreciated.

One thing yo look at in your tests is instabilities in the algorithm, typically caused by the use of hashed collections (eg Set) that have different enumerations from run to run, usually due to hashes being pseudo-randomly computed. If you look at the last commit of the Spur cointerp.c’s you’ll see one variable that changes type from sqInt to usqInt. This points to such a bug. I’ll try and provide you with specifics soon.

Thanks for the warning. I was actually already aware of this bug, but now I also know the cause !

I'm using CCodeGenerator >> #compileToTMethodSelector:in: to get the

...

TMethod, but i cannot seem to find how to use the type inference on this TMethod (or in the instance of CCodeGenerator).

Here's some doit that run Slang including type inference:

https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/Slang%20Tes...

If you use https://github.com/OpenSmalltalk/opensmalltalk-vm/blob/Cog/image/buildspurtr... to build a VMMaker image it'll create an image containing g this and other useful workspaces that contain doit to launch a VM under the simulator, etc.

Type inference is driven by this loop in CCodeGeneraror>>inferTypesForImplicitlyTypedVariablesAndMethods which iterates, inferring types until no more types can b e inferred (until the algorithm has reached a fixed point).

"now iterate until we reach a fixed point" [| changedReturnType | changedReturnType := false. allMethods do: [:m| m inferTypesForImplicitlyTypedVariablesIn: self. (m inferReturnTypeIn: self) ifTrue: [changedReturnType := true]]. changedReturnType] whileTrue.

Type information is derived from explicit return types in methods (the <returnTypeC: > pragma), types for "kernel" sends from CCodeGenerator>>#returnTypeForSend:in:ifNil:, and for types propagated to returns from variables that themselves are either explicitly typed using <var: ... type: ...> and <var: ... declareC: ...> pragmas, or get their types in assignments of the results of methods, where the type they acquire is the return types of those methods.

...

Also, I have been wondering what is the semantic of sqInt. I found out it was an alias for a long, but i couldn't find better than that.

#sqInt is first of all a pun for "squint"

nice , I was wondering if that was the case too :D

, a shortening of Squeak Integer. It is an integer type of the same size as an "ordinary object pointer", or "oop". #usqInt, #sqLong & #usqLong are other important types. #sqLong & #usqLong are 64-bit types. #sqInt & #usqInt are 32-bit types in 32-bit VMs and 64-bit types in 64-bit VMs.

...

Are there offsets to find objects in the object memory?

A #sqInt is typically used to hold the oop for an object and indeed it points to the header word of the object. The object is preceded by an overflow header word if it has more than 254 slots.

...

Are there cast as pointers?

Yes. See the functions and macros in platforms/Cross/vm/sqMemoryAccess.h.

...

Thank you in advance :)

Pierre.

_,,,^..^,,,_

best, Eliot

Cheers

On 18/12/2019 07:36, pierre misse wrote:

On 18/12/2019 03:52, Eliot Miranda wrote:

Hi Pierre,

On Tue, Dec 17, 2019 at 9:50 AM pierre misse <pierre_misse25@msn.commailto:pierre_misse25@msn.com> wrote:

Hi,

I saw my previous mistake, and corrected this. This is a sample test, for a very simple inference.

testReturnFloatConstantNode | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: SlangTypeInferenceTestsClass. ccg inferTypesForImplicitlyTypedVariablesAndMethods. tMethod := ccg methodNamed: #returnAFloatConstantNode.

self assert: tMethod isNotNil. self assert: (ccg typeFor: tMethod statements first in: tMethod) equals: #double. self assert: tMethod returnType equals: #double

And perhaps have SlangTypeInferenceTestsClass inherit from StackInterpreter or InterpreterPrimitives. That will give you much more context.

I was currently inheriting from interpreterPlugin, but I can indeed switch.

My tests currently were as simple as it gets:

constantNode Returm constant return Explicit argument return Explicit Temporary return temp assigned by constant.

and return message send (which returns a constant)*

So it wasn't necessary.

Only surprising result is the return temp assigned by constant, which looses the type. This seems to be easily improvable in the future if you'd like.

returnTempFloatConstantNode | t | t := 1.0. ^ t

testReturnTempFloatConstantNode | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: SlangTypeInferenceTestsClass. ccg inferTypesForImplicitlyTypedVariablesAndMethods. tMethod := ccg methodNamed: #returnTempFloatConstantNode.

self assert: tMethod isNotNil. self assert: (ccg typeFor: tMethod statements first in: tMethod) equals: #double. self assert: (ccg typeFor: tMethod statements first variable in: tMethod) equals: #sqInt. self assert: tMethod returnType equals: #sqInt.

self error "wait what"

I need to work on ones with ambiguity.

The instability I saw was in getErrorObjectFromPrimFailCode; its two locals clone & errObj are sometimes typed as squint and sometimes as usqInt. Some times numPointerSlotsOf ends up with a return type of usqInt, some times as sqInt. I've recently had success in another project substituting OrderedDictionary et al for Dictionary. So I'm sure that solutions can be found easily.

Thanks for the pointer !

Pierre

Hi,

It's been a ... Long couple of months. I wanted to find the time to contribute the few tests I have, but was not able to find time to do so. I join them to this email (lamely) as .st. They are available on my github fork, in the SlangTranslation branch aswell: git@github.com:hogoww/OpenSmalltalk-VMmailto:git@github.com:hogoww/OpenSmalltalk-VM.

I had to move on from them quite quickly so some stuff is very basic, but this is still a start. I tagged with self assert: false the tests results I thought were weird, with a comment of what I expected.

I was not able to look at what you pointed me to, sorry Eliot.

Hope this will help, have a nice day !

Pierre

On 18/12/2019 09:25, Eliot Miranda wrote:

Hi Pierre,

On Tue, Dec 17, 2019 at 10:36 PM pierre misse <pierre_misse25@msn.commailto:pierre_misse25@msn.com> wrote:

On 18/12/2019 03:52, Eliot Miranda wrote:

Hi Pierre,

On Tue, Dec 17, 2019 at 9:50 AM pierre misse <pierre_misse25@msn.commailto:pierre_misse25@msn.com> wrote:

Hi,

I saw my previous mistake, and corrected this. This is a sample test, for a very simple inference.

testReturnFloatConstantNode | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: SlangTypeInferenceTestsClass. ccg inferTypesForImplicitlyTypedVariablesAndMethods. tMethod := ccg methodNamed: #returnAFloatConstantNode.

self assert: tMethod isNotNil. self assert: (ccg typeFor: tMethod statements first in: tMethod) equals: #double. self assert: tMethod returnType equals: #double

And perhaps have SlangTypeInferenceTestsClass inherit from StackInterpreter or InterpreterPrimitives. That will give you much more context.

I was currently inheriting from interpreterPlugin, but I can indeed switch.

I think you have it right. interpreterPlugin should be fine.

My tests currently were as simple as it gets:

constantNode Returm constant return Explicit argument return Explicit Temporary return temp assigned by constant.

So it wasn't necessary.

Only surprising result is the return temp assigned by constant, which loses the type. This seems to be easily improvable in the future if you'd like.

returnTempFloatConstantNode | t | t := 1.0. ^ t

Its its return type isn't inferred to be #double then that's a bug.

testReturnTempFloatConstantNode | ccg tMethod | ccg := CCodeGenerator new. ccg addClass: SlangTypeInferenceTestsClass. ccg inferTypesForImplicitlyTypedVariablesAndMethods. tMethod := ccg methodNamed: #returnTempFloatConstantNode.

self assert: tMethod isNotNil. self assert: (ccg typeFor: tMethod statements first in: tMethod) equals: #double. self assert: (ccg typeFor: tMethod statements first variable in: tMethod) equals: #sqInt. self assert: tMethod returnType equals: #sqInt.

Well, t is untyped and it is assigned a double. So I would expect that the inferencer infer the ty[e #double for f. Then it should infer the type #double for returnTempFloatConstantNode since it answers t.

self error "wait what"

I need to work on ones with ambiguity.

The instability I saw was in getErrorObjectFromPrimFailCode; its two locals clone & errObj are sometimes typed as squint and sometimes as usqInt. Some times numPointerSlotsOf ends up with a return type of usqInt, some times as sqInt. I've recently had success in another project substituting OrderedDictionary et al for Dictionary. So I'm sure that solutions can be found easily.

Thanks for the pointer !

Pierre

Thanks for looking!! It will be great to have tests for this, if only for good documentation. I never had the time, and always look at the diffs of the generated code. But that doesn't mean I don't want these tests. Thank you for your energy!

_,,,^..^,,,_ best, Eliot