Am Sunday 20 February 2005 02:10 schrieb Russell Penney:
Now one of my biggest concerns is that we should try to develop EVERYTHING in Squeak with as few plugins as possible. I wrote the OGG decoder in "pure" Smalltalk so it is possible. Of course by converting a small part of the decoder to a plugin I sped everything up 10 fold but that came later ;)
I think this is important to remove platform dependencies or dependencies on external libraries. We may end up using them purely for performance but I think we should have a goal of not using them in the first part.
I don't share your point of view here. Writing codecs in pure smalltalk is interesting and may teach you a lot. But I don't think we have the manpower to do this. I want to have support for mpeg2, mpeg4 (xvid, divx), theora, etc, and windows people may want to use the microsoft codecs. Do you want to implement all this again - if it were possible at all. And I think the speed of a codec is very important. Every cpu cycle counts.
I would like to have small Class for each codec that uses the FFIPlugin as interface to the external library. This way everybody with a basic knowledge of his favourite codecs API could integrate it integrate it in no time.
We should concentrate on the container stuff and nice guis to manipulate the basic multimeda entities. This is work enough for the next month.
Martin
Martin Kuball MartinKuball@web.de wrote:
were possible at all. And I think the speed of a codec is very important. Every cpu cycle counts.
I would like to have small Class for each codec that uses the FFIPlugin as interface to the external library.
Two conflicting requirements there; FFI is not a particularly fast way to access a plugin. If every cycle counts, a custom plugin Slang section is the way to go. By all means do the initial experiments via FFI but if you want the fastest prim access, sling dat Slang.
By the way, in the latest VMs the address of the external prim is cached in the method lookup table. Once they are in the cache they should be exactly as fast to get to as normal prims.
tim -- Tim Rowledge, tim@sumeru.stanford.edu, http://sumeru.stanford.edu/tim Any program that runs right is obsolete.
Am Sunday 20 February 2005 19:45 schrieb Tim Rowledge:
Martin Kuball MartinKuball@web.de wrote:
were possible at all. And I think the speed of a codec is very important. Every cpu cycle counts.
I would like to have small Class for each codec that uses the FFIPlugin as interface to the external library.
Two conflicting requirements there; FFI is not a particularly fast way to access a plugin. If every cycle counts, a custom plugin Slang section is the way to go. By all means do the initial experiments via FFI but if you want the fastest prim access, sling dat Slang.
Well, I guess I was exagerating a little bit here. For each frame you want to encode or decode you have normally one library call. The processing inside the lib takes on the order of some milliseconds For NTSC it should be less than 40 if you want to have real time. So what time does FFI add to this?
By the way I thought that croquet uses FFI to to the OpenGL access.
Martin
Martin,
Well, I guess I was exagerating a little bit here. For each frame you want to encode or decode you have normally one library call. The processing inside the lib takes on the order of some milliseconds For NTSC it should be less than 40 if you want to have real time. So what time does FFI add to this?
By the way I thought that croquet uses FFI to to the OpenGL access.
I guess the bigger issue is where the "execution state" is maintained. With FFI, Squeak side cannot do too much. You probably would like to start a few threads outside of Squeak to decode the video stream, but it would be really cumbersome to control these threads via FFI. (Imagine to write SocketPlugin without C code support. or a multi person (n > 2) video conferencing seesion of some kind.)
I think it makes more sense to provide semi-platform independent layer in a plugin, and use it from Squeak.
Yes, Brad, I'd be interested in the multimedia group.
-- Yoshiki
Hi,
I had some success with Yoshiki's 3.4.1 Qtopia sources and the bitbake (aka openembedded) build system for handhelds. I still have to finish the ipkg creation but so far I got
14250000 bytecodes and 470000 sends per second on a 206 MHz ARM using some 3.4 gcc. This is a 50% increase compared to gcc 2.95. If no one objects (and if the OpenEmbedded guys want it), I would like to commit this squeak-nox to the build system with a hope of an automatic embedded Squeak. It should at least run on Ipaq and Simpad.
Torsten
P.S. I had to do some hacks to get it built. I would appreciate if some concessions could be made to cross compiling. Especially configure and #include "aio.h" (nameclash with system aio.h) are troublesome.
Torsten Sadowski moehl@akaflieg.extern.tu-berlin.de wrote:
14250000 bytecodes and 470000 sends per second on a 206 MHz ARM using some 3.4 gcc. This is a 50% increase compared to gcc 2.95.
That's quite good; even a little faster than my old 206MHz StrongARM 110 Acorn machine. Nice to see gcc has finally caught up for ARM code.
P.S. I had to do some hacks to get it built. I would appreciate if some concessions could be made to cross compiling. Especially configure and #include "aio.h" (nameclash with system aio.h) are troublesome.
I strongly suggest getting in touch directly with Ian to work on this.
tim -- Tim Rowledge, tim@sumeru.stanford.edu, http://sumeru.stanford.edu/tim Useful random insult:- His head whistles in a cross wind.
Torsten,
14250000 bytecodes and 470000 sends per second on a 206 MHz ARM using some 3.4 gcc. This is a 50% increase compared to gcc 2.95. If no one objects (and if the OpenEmbedded guys want it), I would like to commit this squeak-nox to the build system with a hope of an automatic embedded Squeak. It should at least run on Ipaq and Simpad.
That is good news. I should check out if the SL-series Zaurus can benefit from the new gcc.
-- Yoshiki
You want to use plugins, sure go ahead, I want to use smalltalk, sure go ahead. If we develop a framework correctly that lets us mix and match, why not do it that way? You want to have support for MPEG2 and 4 and theora? Great! So do I, but I think you are missing the point that there will be a LARGE number of simple codecs. A lot of these already have most if not all of their code in Squeak which can be re-used and/or ported to a new framework so that we have consistency. This framework will allow you to use FFI or Plugins or Smalltalk or some new technique. Your class can have as much functionality as you want and as long as it conforms to the minimum spec, it will auto magically work with any of the tools. These simple codecs become the core, platform independent set of media types we distribute in Base. External libraries are great and make perfect sense not having to reinvent the wheel but they also have major limitations. As an example, can you play an MP3 using the Squeak classes from a Squeak socket? I know I could with the OGG library because it wanted data from a FILE structure. I am sure there is some tricky way to do that but I was easier for me to re-write it in Smalltalk than cobble together some badly integrated solution.
The problem with trying to do the containers now is that there are so many exceptions because of the haphazard way the code has evolved. GUIs are great except when there isn't a consensus on how the data is presented which is the situation now. Try to write a tool that plays any audio type in Squeak from a URI, very tricky at the moment :)
So to recap so people don't get the wrong idea. I don't have a problem with plugins/external libraries. They are necessary for some codecs but there are Lots of others where a plugin is overkill. You might as well just write everything in "C" ;)
I know it is boring but getting the base stuff sorted (as much as we can at the moment) means lots less effort later. Oh and it means that there is at least one set of tests that works for every codec. That saves LOTS of time ;)
-----Original Message----- From: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev- bounces@lists.squeakfoundation.org] On Behalf Of Martin Kuball Sent: Monday, 21 February 2005 4:31 AM To: squeak-dev@lists.squeakfoundation.org Subject: Re: Improving Squeak's Mutlimedia
Am Sunday 20 February 2005 02:10 schrieb Russell Penney:
Now one of my biggest concerns is that we should try to develop EVERYTHING in Squeak with as few plugins as possible. I wrote the OGG decoder in "pure" Smalltalk so it is possible. Of course by converting a small part of the decoder to a plugin I sped everything up 10 fold but that came later ;)
I think this is important to remove platform dependencies or dependencies on external libraries. We may end up using them purely for performance but I think we should have a goal of not using them in the first part.
I don't share your point of view here. Writing codecs in pure smalltalk is interesting and may teach you a lot. But I don't think we have the manpower to do this. I want to have support for mpeg2, mpeg4 (xvid, divx), theora, etc, and windows people may want to use the microsoft codecs. Do you want to implement all this again - if it were possible at all. And I think the speed of a codec is very important. Every cpu cycle counts.
I would like to have small Class for each codec that uses the FFIPlugin as interface to the external library. This way everybody with a basic knowledge of his favourite codecs API could integrate it integrate it in no time.
We should concentrate on the container stuff and nice guis to manipulate the basic multimeda entities. This is work enough for the next month.
Martin
squeak-dev@lists.squeakfoundation.org
-
Martin Kuball -
Russell Penney -
Tim Rowledge -
Torsten Sadowski -
Yoshiki Ohshima