Hi Jakob

> Do you think that the dimensions are always known?

Yes, how would you else be able to write an FFI interface in the first place? If an interface says "int**" and documents "can be int***" from time to time, then I hope it does also give a hint on how to find that out. Is that even possible with C compilers? From within Squeak, it's not an issue to add that extra dimension dynamically (i.e. just switch the external type for interpretation from, e.g., int** to int***) when trying to read the data. :-)

> Note that an int** is not a two-dimensional array int[x][y], so it might be misleading to speak of dimensions.

I don't think it makes a difference from the Squeak FFI perspective. Pointer arithmetic for such access is currently implemented in ExternalData >> #at: and #at:put:. I don't think we should use more new terminology than necessary.

> If you want to remember only the number of dimensions, some of my remarks may not apply.

Of course, only the number of dimensions. The length/size has to be provided somewhere else. Maybe another field in my external struct. :-) Or maybe zero-terminatd if the library's documentation claims so. Then I have to count manually and store it in ExternalData >> #size. After that, I can enumerate the data.

> For more fun

Thank you for the examples!

Best,
Marcel
Am 14.06.2020 21:14:20 schrieb Jakob Reschke <forums.jakob at resfarm.de>:
Hi Marcel,

Do you think that the dimensions are always known? I understood that you also need to know the cardinality per dimension of the involved arrays. If you want to remember only the number of dimensions, some of my remarks may not apply. Note that an int** is not a two-dimensional array int[x][y], so it might be misleading to speak of dimensions. From practice I know we would need to support at least three pointer indirections, not only two, at least if you don't want to require intermediate type aliases for pointers in some cases. See below.

Look at the CredRead function and CREDENTIALS struct for example:
https://docs.microsoft.com/en-us/windows/win32/api/wincred/nf-wincred-credreadw [https://docs.microsoft.com/en-us/windows/win32/api/wincred/nf-wincred-credreadw]
https://docs.microsoft.com/de-de/windows/win32/api/wincred/ns-wincred-credentialw [https://docs.microsoft.com/de-de/windows/win32/api/wincred/ns-wincred-credentialw]

The CredRead function takes essentially a CREDENTIAL** as an output parameter, so it will give you (dereferencing the passed argument) one CREDENTIAL*. It is like passing a PCREDENTIAL[1] variable, but you could well pass it a PCREDENTIAL[6] if you like, since both decay to PCREDENTIAL* when passed as arguments. Anyway, this is not a two-dimensional array of CREDENTIALs.

For more fun, consider CredEnum: https://docs.microsoft.com/en-us/windows/win32/api/wincred/nf-wincred-credenumeratew [https://docs.microsoft.com/en-us/windows/win32/api/wincred/nf-wincred-credenumeratew]
It takes a CREDENTIAL*** to give you back an array of pointers to CREDENTIAL structs. How many there are, you will only learn by dereferencing the count argument. The actual CREDENTIAL structures /could/ be laid out as a CREDENTIAL[count] array, but they don't need to be (implementation detail). So there is a one-dimensional array of pointers, there could be, but needn't be, a one-dimensional array of CREDENTIALs, but certainly there are no two- or three-dimensional arrays here.

Kind regards,
Jakob


Am So., 14. Juni 2020 um 18:19 Uhr schrieb Marcel Taeumel <marcel.taeumel at hpi.de [mailto:marcel.taeumel at hpi.de]>:

Hi all!

For some days now, I am thinking about "pointer-to-pointer" types. I suppose that the more general topic is the interpretation of multi-dimensional arrays.

On the one hand, this is an in-image issue to correctly read ExternalData and to generate appropriate accessors for struct fields.

On the other hand, this may affect FFI call's arg coercing and return-type packaging.

So, after playing around with an implementation of pointer-to-pointer types via ExternalType's #referencedType (i.e. making it a chain of three: ... -> void -> void* -> void** -> void ...), I figured that we might need to store the array dimensions in the compiledSpec.

Here are some unused bits for that:

FFIFlagPointerArrayMask := 16rF00000. "up to 16-dimensional arrays"
FFIFlagPointerArrayShift := 20.

Here are some questions for you on this topic:

- Should we reserve all 4 bits? 16-dimensional arrays sound like overkill ... 2 bits could be enough, having rarely the lower one set for char** ...
- In the image, would you store all possible versions through refrencedType in a linked cycle? Or would you lazily create them ... on the fly? As requested from struct fields and FFI call specs via ExternalType class >> #typeNamed: ?
- In the FFI plugin, do you see value in coercing an array of, for example, IntegerArrays for an FFI call? (int**)
- In the FFI plugin, do you see value in automatically packaging returned objects if those dimensions would be zero-terminated?

Best,
Marcel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200615/f4503bf6/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 15519 bytes
Desc: not available
URL: <http://lists.squeakfoundation.org/pipermail/squeak-dev/attachments/20200615/f4503bf6/attachment.png>