[Vm-dev] Re: [Pharo-dev] Cog in the cloud

[Ben Coman]

Ben Coman wrote:
> 
> (whoops forgot to cross-post)
> 
> Eliot Miranda wrote:
>> Hi Ben,
>>
>>
>> On Dec 19, 2014, at 8:08 PM, Ben Coman <btc at openInWorld.com> wrote:
>>
>>> Just some thoughts that arose as I skipped along the web...
>>>
>>> With the idea of using Pharo in the cloud, I was thinking of how 
>>> PharoNOS[1] is basing off top of the Linux kernel, but the examples 
>>> indicate its operating in User Mode.  So I was wondering about 
>>> performance being greater[2] if the CogVM ran in Kernel Mode. However 
>>> maybe this could get tangled in the GPL license of the kernel.
>>>
>>> Now I believe FreeBSD/NetBSD license is compatible with COG's MIT 
>>> license, so that seems a better option for eliminating layers of the 
>>> operating system.  They both[3][4] have pre-built Amazon Machine 
>>> Images to run on EC2, as well as instructions[5] to build your own 
>>> AMI.  So potentially we could build an AMI with the CogVM linked to 
>>> the FreeBSDKernel operating in Kernel Mode, with no User Mode. The 
>>> lack of direct access to memory from the Image, and probably single 
>>> application focus should be sufficient security to forgo User Mode, 
>>> and run faster.
>>>
>>> But we could go a step further.  Runing in the cloud relies heavily 
>>> on virtualisation, and for performance[6], most likely on the OS's 
>>> paravirtualisation interface - to Xen for example.  So from Cog, why 
>>> not interface direct to the Xen FrontEndDriver[7] and eliminate the 
>>> DomU operating system all together.  What I understand from [6] is 
>>> that Xen's paravirtualisation hooks makes it much easier to boot, 
>>> than booting on the bare metal of a fully-virtualised system. So the 
>>> former seems more achievable than the latter (which was the only 
>>> option in years past), and also be more portable as technology 
>>> evolves, to any platform Xen runs on.
>>>
>>> Squeak/Pharo on Cog-Xen could be a good choice for a Cloud Operating 
>>> System[8]. Maybe a good student research project?
>>
>> This sounds really cool, but could you analyse a bit further?
>>
>> - what key economic and managerial benefits accrue from this 
>> configuration?
> 
> I'll put some more thought into this, but just off the top of my head
> (so it turned out more technical than economic/managerial):
> 
> * Eliminates performance overhead of system call context switching.
> Although this is replaced by hypercalls, it removes one layer on
> hypervisor systems, which plausibly is going to be EVERYTHING in the 
> future.
> 
> * Eliminate performance overhead of OS networking stack
> http://roscidus.com/blog/blog/2014/08/15/optimising-the-unikernel/
> 
> * Data from virtual devices is event driven.
> 
> * More easily portable to different ARM platforms.  Xen will present a
> common interface to hide platform idiosyncrasies.
> --> More widespread embedded use Cog.
> http://www.slideshare.net/xen_com_mgr/free-rtos-xensummit
> 
> * Run databases in their own domain.
> 
> * Isolate plugin memory space. Rather than linking plugins into Cog,
> potentially threatening its stability, run them in a separate domain and
> use high-speed inter-domain shared memory.
> http://www.cs.ucsb.edu/~nanli/projects/cs270.pdf
> 
> * Have a cloud community of thousands** of Images running each in their
> own domain, communicating with high-speed inter-domain shared memory.
>    ** presumptive figure - would need to check scaling ability
> 
> * Data sharing on high density hosting providers. Not sure if its there
> yet, but may get Copy-On-Write
> http://www-archive.xenproject.org/files/xensummit_fall07/18_GregorMilos.pdf
> 
> * In very large Images, which Spur will allow, perhaps a better way to
> snapshot an Image is via virtual machine snapshotting using Copy-On.
> http://www.cercs.gatech.edu/tech-reports/tr2010/git-cercs-10-05.pdf
> 
> * Rather than convert the Cog/Image to be multi-threaded, could have
> many grid nodes running their own domain, communicating with high speed
> shared memory.
> http://www.cs.ucsb.edu/~nanli/projects/cs270.pdf
> http://osnet.cs.binghamton.edu/publications/hines07memx.pdf
> 
> * AMD's hardware virtualisation eliminated a Ring and also segmented
> memory protection used by virtualisation to separate address spaces, so
> 64-bit AMD can be slower.  Eliminating the OS kernel layer probably
> bypasses this limitation since there would be only two levels -
> Hypervisor & Cog.
> https://lse.epita.fr/data/2011-lse-summer-week/xen.pdf
> 
> * Its where all the cool kids hang out --> marketing opportunity
> 

* Security versus lines of code...

   Linux = 15.9 million (3.6 Kernel)
   OSX = 86 million
   FreeBSD = 6.25 million
   OpenSolaris = 10 million
   Windows = 64 million

   Xen-ARM = 90,000
   Xen-x86 = 140,000

https://www.google.com/fusiontables/DataSource?docid=12Eg7Ru1CH5MIjomcfjaGKe2qBd9ynmSQni5CAioK

http://wiki.xen.org/wiki/Xen_ARM_with_Virtualization_Extensions_whitepaper





> 
> 
>>
>> - what infrastructure is missing?  (e.g. what key drivers need 
>> implementing?)
> 
> I'll need to understand Cog more. As a start, here are some porting and
> API docs...
> 
> http://events.linuxfoundation.org/sites/events/files/slides/Porting%20FreeBSD%20on%20Xen%20on%20ARM%20.pdf 
> 
> 
> http://oss.org.cn/ossdocs/server_storage/xen/interface/interface.html
> 
> http://www.informit.com/articles/article.aspx?p=1160234&seqNum=4
> 
> https://lse.epita.fr/data/2011-lse-summer-week/xen.pdf
> 
> https://github.com/cloudius-systems/osv/blob/master/drivers/xenfront-xenbus.cc 
> 
> 
> cheers -ben
> 
>>>
>>> [1] http://pillarhub.pharocloud.com/hub/mikefilonov/pharonos
>>> [2] http://blog.codinghorror.com/understanding-user-and-kernel-mode/
>>> [3] http://www.daemonology.net/freebsd-on-ec2/
>>> [4] http://wiki.netbsd.org/amazon_ec2/amis/
>>> [5] http://wiki.netbsd.org/amazon_ec2/build_your_own_ami/
>>> [6] http://wiki.xen.org/wiki/Virtualization_Spectrum
>>> [7] http://wiki.xen.org/wiki/FrontendDriver
>>> [8] http://wiki.xen.org/wiki/Cloud_Operating_Systems
>>
>>
>> Eliot (phone)
>>
> 
> 
> 
>