On 03-07-2013, at 11:13 AM, Chris Muller <asqueaker at gmail.com> wrote:

> Good discussion thanks Levente.  You know, after more thinking I had
> trouble understanding what is the characteristic of the FilePlugin
> primitives that leads to better performance in a BFS?

Just take a look at the computational gymnastics undertaken in the lowest level of FilePlugin code; both in the image and the actual platform code. For all platforms it is a nightmare of untangling, mangling and re-tangling strange values that bear little relationship to much useful. As an example, look at what happens when  dir_Lookup is used in unix. Eeek!

I suggest that the platform handling is done at far too low level. Different platforms do things rather differently and rather than the current pattern of squeezing all of them into a straitjacket - where dir_Lookup is an example - we ought to have a higher level fanout. Keeping to unix as an example, consider what happens in a modest sized directory when running FileDirectory>entries (and we should temporarily ignore the ridiculous uses that gets put to); #directoryContentsFor:do: repeatedly calls the primitive that then uses dir_lookup. By the time you get to looking up entry 42 in the directory you would have already done around 860 readdir calls if there wasn't a sneaky optimisation that keeps the last opened dir stream around to see if it is the next needed  one. When doing a BFS, that is generally true, thus some benefit accrues.

If we had image level code that understood how readdir works and took advantage of it through a suitable prim we might gain some interesting advantages.Likewise for the specific apis on other platforms; RISC OS for example can look directly at entry(i) but also has to do terrible things to handle the idiotic edge case of passing an empty string argument to mean 'list all the roots'. Take a look at dir_LookupRoot in platforms/RiscOS/plugins/FilePlugin/sqRPCDirectory.c and reel in shock!

tim
--
tim Rowledge; tim at rowledge.org; http://www.rowledge.org/tim
"How many Ringworld Engineers does it take to change a lightbulb?" "Thirty. Hey, moving suns around isn't easy..."