[Vm-dev] Issue 99 in cog: Link LZ4 Compression

[Camillo Bruni]

>>> I do not understand what this means. Is it a request for someone to
>>> write a plugin?
>> Link cog with the lz4 as a first step to write bindings with FFI/NativeBoost.
> 
> An external plugin sounds a lot better to me.
replace "link" with "bundle".

> The larger the VM binary is, the slower it will be on today's CPUs.

how come?

> Make it work, make it right, make it fast. You don't have the system yet, but you want to make it fast already?

I agree 100% if you implement everything from scratch by yourself.
But in this case it's relying on an external project, which will give me speedup for free ;).


>> Plus by having a super-fast compression library at hand decompression would
>> essentially be a NOP.
>> 
> 
> I didn't see any benchmarks where (de)compression is done on small chunks of data (a few kilobytes at most - which is your intended use case).

see [1]

> And even though the (de)compression might not make much difference in runtime, it definitely will give higher CPU usage, which is unwelcome in some cases (e.g. mobile devices). 

well it runs on multiple cores. Cog runs on a single core. So wasting some CPU cycles
on the non-used cores won't harm that much.

For mobile devices you might simply not want the image to swap, hence you will pay
a lot of attention to make sure it stays small. So yes, in this case you won't rely
directly on such a features. 

However swapping out unused parts of the system and reload them are still interesting
on such a "limited" platform  [1]. And in this case you exactly don't want to waste 
cycles on loading the data, so compression in memory is interesting again.

> It might result in lower overall CPU usage too, but the ~2 compression ration makes me think that it's unlikely.


It does if it makes swapping out data cheaper, that's a win. But here you don't 
show me benchmarks either ;) 


[1] http://rmod.lille.inria.fr/archives/papers/Mart11c-COMLAN-ObjectSwapping.pdf