Hi All,
Fantastic, we have several people who are interested and I hope willing to play a part in improving Squeak's Multimedia capabilities:
Blake (Blake: what's your last name?)
Dean Swan
Craig Latta
Russell Penney
Steve Elkins
Martin Kuball
Yoshiki Ohshima
Mark McCahill
Juan Vuletich
Cornelius Huber
Brad Fuller
I hope I didn't misspell or miss someone. If so, apologies.
Let me start off the discussion by asking some hi-level questions. And, by all means, please chime in. I present only one slanted POV --- your views will be valuable to provide a robust analysis of the situation.
=== Multimedia Definition
Maybe the first order of business is to achieve agreement on what we mean by "Multimedia".
I looked it up after Martin's email and he's right - multimedia includes a variety of media such as the real-time areas (e.g. film/video, music) as well as static media (e.g. text, graphics.)
Russell provided another interesting POV that containers could hold a multitude of multimedia assets. and that these basic frameworks could provide easy re-use.
My 2cents: I was only thinking of "real-time" assets such as audio, music, video, etc.
What are your thoughts on the definition of Multimedia?
=== Hi Level Interests?
Furthermore, our definition of Multimedia may be a clear superset of what we want to tackle. For instance, it might turn out that we have just enough hands, or interest, to look at the audio classes; or maybe just video; or maybe the majority is only interested in adding MPEG4 to Squeak.
For me, I'm obviously interested in only audio and video. I know this is broad, but I just want to get the top level issues out of the way. As an example of elimination, I am not interested in graphics or text.
What are your interests?
==== Attack Plan
Finally, I propose that we decide on what to tackle and how to do it.
From a bird's eye view, we might look at 3 distinct phases (not necessarily linear in order - can be approached simultaneously):
* Review/Discuss/Plan/Fix existing classes (which hopefully fix existing tools/apps) * Discuss-Design/Plan/Enhance existing classes * Discuss-Design/Plan/Add new capabilities: new classes, tools and end-user applications.
(That's a lot, right there.)
I'm willing to put in time, at the very least, toward fixing and enhancing the audio/music side of MM. I'm also interested in helping to test and use any video functionality - especially audio-video synchronization issues. Lastly, I would like to begin discussing new real-time capabilities for Squeak -- sort of plan while we get the house in order (the discussions might circumvent fixes, who knows.)
What areas interest you?
And, are you interested in helping?
===== Is This A Team?
Finally, Finally,
In light of the current squeak-dev traffic: Does this look like a team to propose? It seems orthogonal to the team proposals from the castaways.
What do you think about MM as a team? Is it a sub-team? How does the rest of the Squeak community feel?
brad
BTW: Are there any related on-going projects that should be incorporated?
On Tue, 22 Feb 2005 12:35:40 -0800, Brad Fuller brad@sonaural.com wrote:
Hi All,
Fantastic, we have several people who are interested and I hope willing to play a part in improving Squeak's Multimedia capabilities:
Blake (Blake: what's your last name?)
Watson
I looked it up after Martin's email and he's right - multimedia includes a variety of media such as the real-time areas (e.g. film/video, music) as well as static media (e.g. text, graphics.)
Well, y'know, "multimedia" is just a buzzword. (A ten year-old buzzword, to boot.) "Multimedia" applies to using several media in one context, like an encyclopedia entry on Bach that plays "Cantata no. 157" and shows video of his 50th birthday.
Arugably, what we're interested in is A/V support. (Some of us the A, some of us the V.<s>)
For me, I'm obviously interested in only audio and video. I know this is broad, but I just want to get the top level issues out of the way. As an example of elimination, I am not interested in graphics or text.
I wasn't aware Squeak needed much help in the graphics/text area.
What are your interests?
I'd like to see an equalizer. I think it'd also be cool if Squeak could be used to teach orchestration/composition. And video editing. And...<s>
I don't know much about the media formats themselves, though. My contributions would most likely be: I can test, I can document, and I can build end-user stuff.
Am Tuesday 22 February 2005 22:39 schrieb Blake:
On Tue, 22 Feb 2005 12:35:40 -0800, Brad Fuller brad@sonaural.com
wrote:
Hi All,
Fantastic, we have several people who are interested and I hope willing to play a part in improving Squeak's Multimedia capabilities:
Blake (Blake: what's your last name?)
Watson
I looked it up after Martin's email and he's right - multimedia includes a variety of media such as the real-time areas (e.g. film/video, music) as well as static media (e.g. text, graphics.)
Well, y'know, "multimedia" is just a buzzword. (A ten year-old buzzword, to boot.) "Multimedia" applies to using several media in one context, like an encyclopedia entry on Bach that plays "Cantata no. 157" and shows video of his 50th birthday.
Arugably, what we're interested in is A/V support. (Some of us the A, some of us the V.<s>)
For me, I'm obviously interested in only audio and video. I know this is broad, but I just want to get the top level issues out of the way. As an example of elimination, I am not interested in graphics or text.
I wasn't aware Squeak needed much help in the graphics/text area.
IMHO Squeak already has a lot of things we can use. But they need to be better integrated, polished up a little bit and bundled with some nice interface classes.
Martin
On Tue, 22 Feb 2005 23:25:21 +0100, Martin Kuball MartinKuball@web.de wrote:
IMHO Squeak already has a lot of things we can use. But they need to be better integrated, polished up a little bit and bundled with some nice interface classes.
That's my feeling, too.
Blake wrote:
On Tue, 22 Feb 2005 23:25:21 +0100, Martin Kuball MartinKuball@web.de wrote:
IMHO Squeak already has a lot of things we can use. But they need to be better integrated, polished up a little bit and bundled with some nice interface classes.
That's my feeling, too.
Yes, but maybe more than polished. Some things just don't work.. or they work on some platforms and not on others. Or, they are not clean objects to build on.
At the very least, it'd be great to get these fixed/polished/bundled/interfaced/documented to use. That would open up for more enhancements by others. I'd like to make Squeak more attractive to other audio/video developers, too.
brad
My replies to various comments to the list (I have been on a training course the last couple of days):
My main interest is making sure I can use any format I want in any of the tools that will be designed. I have been collecting format specification documents for a while now and am more than happy to help in that area.
On Tue, 22 Feb 2005 23:25:21 +0100, Martin Kuball MartinKuball@web.de wrote:
IMHO Squeak already has a lot of things we can use. But they need to be better integrated, polished up a little bit and bundled with some nice interface classes.
That's my feeling, too.
Yes, but maybe more than polished. Some things just don't work.. or they work on some platforms and not on others. Or, they are not clean objects to build on.
At the very least, it'd be great to get these fixed/polished/bundled/interfaced/documented to use. That would open up for more enhancements by others. I'd like to make Squeak more attractive to other audio/video developers, too.
Exactly!! ImageReadWriter and subclasses are great but don't go far enough (IMHO). The audio classes are half done (some work really well and some really don't). If you want to add another file format type you might have to write a BUNCH of methods in lots of classes to integrate it. We need to come up with a framework that allows a new format to be EASILY added. i.e. registration, read from stream, write to stream, etc.
BTW I have a vision of a framework, not the details so much as how I would like it to work. I will write it up soon. I fly to Malaysia on Sunday for a couple of weeks so I should have lots of time in the hotel :)
Cees said:
Good. As a hobbyist photographer (I think stills are part of multimedia as well) and musicist I welcome any enhancement in that area (one of my todo list entries shows a media database - I want to replace iTunes *and* my photo db software with Squeak ;)).
One of my dreams is to have Squeak be my music/video/photo database. That is why things like metadata and containers are important for me. i.e EXIF in JPEG
Yoshiki Said:
My interest so far is rather specific. Adding MPEG4 to Squeak,
- It should handle multiple (network and file) streams.
- It should reasonably portable.
My thought now is to write an equivalent of mp4player in Squeak, and call
MPEG4IP library as plugin.
I haven't looked at OGG thing, but possibly the higher level abstraction
can use it...
The OGG stuff is sorted, it needs better integration with sound classes and usability methods but the core works. I stopped there because when I came to actually play the sounds, the audio classes did my head in!! And this was AFTER figuring out how to port something where I didn't really understand the underlying math! ;)
I will make my feelings about plugins clear for the last time (cause no one like a nagger :). I see plugins as a necessary evil. Yes they speed things up but they ALSO reduce portability. I would love to see a "pure" smalltalk implementation of all A/V classes with plugins used to speed up parts that are required. That way the base is portable to any machine, it might not be fast but it will work.
If we make the assumption that the A/V framework will rely on binary streams, we separate the transport layer from the encoding/decoding layer (which should be separate from the presentation layer). Something like: Stream<->Codec<->Tools
Russell
On Tue, 22 Feb 2005 12:35:40 -0800, Brad Fuller brad@sonaural.com wrote:
Fantastic, we have several people who are interested and I hope willing to play a part in improving Squeak's Multimedia capabilities:
Good. As a hobbyist photographer (I think stills are part of multimedia as well) and musicist I welcome any enhancement in that area (one of my todo list entries shows a media database - I want to replace iTunes *and* my photo db software with Squeak ;)).
Do you want a mailing list to go with that group? If so, just drop me a mail with the proposed name.
Cees de Groot wrote:
On Tue, 22 Feb 2005 12:35:40 -0800, Brad Fuller brad@sonaural.com wrote:
Fantastic, we have several people who are interested and I hope willing to play a part in improving Squeak's Multimedia capabilities:
Good. As a hobbyist photographer (I think stills are part of multimedia as well) and musicist I welcome any enhancement in that area (one of my todo list entries shows a media database - I want to replace iTunes *and* my photo db software with Squeak ;)).
Do you want a mailing list to go with that group? If so, just drop me a mail with the proposed name.
Thanks for the offer Cees. My input is that we should get some buy-in and agreement first and then move on to getting things done. A mailing list might be appropriate to keep the SNR appropriate for the main squeak-dev ml.
Interesting ... replacing iTunes. Do you mean to add the licensing functionality and connect to the site for downloads? Or just a player ala xmms?
brad (what is a musicist?)
On Tue, 22 Feb 2005 16:30:32 -0800, Brad Fuller brad@sonaural.com wrote:
Interesting ... replacing iTunes. Do you mean to add the licensing functionality and connect to the site for downloads? Or just a player ala xmms?
Playing MP3's is simple. Keeping them organized is the hard part :). Of all the players I know, iTunes is best here, but there's lots of room for optimization. Ditto for the photo db management programs (and now here's a market - lots and lots of people running around with digital cameras suddenly having to manage gigabytes of data. The tools available are either shit or tailored for professional use).
brad (what is a musicist?)
Someone who doesn't know what a musician is? ;)
On Tue, 22 Feb 2005 12:35:40 -0800, Brad Fuller brad@sonaural.com wrote:
What are your thoughts on the definition of Multimedia?
My definition is quite broad. I'm more interested in (forming a better understanding of) Squeak's ability to integrate various media.
What are your interests?
For example, I make up stories for my daughter (7). She likes to draw pictures in Squeak. I want to record stories and have her illustrate them, like narrated slide shows. I bet I could do this already. I also want to set stories to music.
What areas interest you?
Testing and using for now. If I develop an itch, I'll scratch it.
And, are you interested in helping?
Yes.
What do you think about MM as a team? Is it a sub-team?
I don't know, need to understand 'team' better, or accept different senses of the term. I have quite a few recent threads flagged for careful perusal.
BTW: Are there any related on-going projects that should be incorporated?
Probably, don't know.
Regards, Steve
Hi people!
Steven Elkins sgelkins@gmail.com wrote:
What do you think about MM as a team? Is it a sub-team?
I don't know, need to understand 'team' better, or accept different senses of the term. I have quite a few recent threads flagged for careful perusal.
The current table of Teams and the description of the model:
http://anakin.bluefish.se/castaways/6
As to the question if this could be a Team or not - IMO it can definitely be a Team.
*Personally* I consider a Team to be a group of Squeakers cooperating towards one or more specific goals. Not more complicated than that really.
A description of the goals and someone standing up ready to take the Team leader role is more or less what is needed - we, the Coordinators - want to encourage teams so even if we also have a part on our side (see the model) we are willing to take that on.
I mean, what is the worst thing that can happen? It is *not* the Team failing, the worst thing that could happen is if the members of the Team somehow end up feeling disappointed at either the Team or us.
We will work hard to prevent the last thing to happen, the leader will have to prevent the first thing.
So in short - a Team is allowed to fail - just as long as you do it with a smile and get back up on your feet. :)
regards, Göran
Brad, I'm interested in being a tester. Maybe I can contribute to development at some point, after I learn more about the domain...
I'm really excited and encouraged by your goals for MM, and the response you're getting - I think it would be *great* for Squeak if it became known as an excellent tool for playing (and working) with audio/music/video.
FWIW I think you are doing a very good job of building a Team.
Hello,
What are your interests?
My interest so far is rather specific. Adding MPEG4 to Squeak,
* It should handle multiple (network and file) streams. * It should reasonably portable.
My thought now is to write an equivalent of mp4player in Squeak, and call MPEG4IP library as plugin.
I haven't looked at OGG thing, but possibly the higher level abstraction can use it...
-- Yoshiki
Hi Yoshiki,
Are you focused only on MPEG4 playback, or also encoding? How goes the battle?
Josh
Yoshiki Ohshima wrote:
Hello,
What are your interests?
My interest so far is rather specific. Adding MPEG4 to Squeak,
- It should handle multiple (network and file) streams.
- It should reasonably portable.
My thought now is to write an equivalent of mp4player in Squeak, and call MPEG4IP library as plugin.
I haven't looked at OGG thing, but possibly the higher level abstraction can use it...
-- Yoshiki
Brad Fuller wrote:
==== Attack Plan Finally, I propose that we decide on what to tackle and how to do it.
From a bird's eye view, we might look at 3 distinct phases (not necessarily linear in order - can be approached simultaneously):
- Review/Discuss/Plan/Fix existing classes (which hopefully fix existing tools/apps)
- Discuss-Design/Plan/Enhance existing classes
- Discuss-Design/Plan/Add new capabilities: new classes, tools and end-user applications.
And, are you interested in helping?
As mentioned in my msg, I'm interested in mainly fixing and enhancing the audio base classes, first.
From the comments so far, we don't a clear winner -- not any particular area that everyone feels compelled to work on. Nothing jumps out at me. Anything jump out at you?
Here's what I have so far:
Blake Watson
Interests:
I'd like to see an equalizer. I think it'd also be cool if Squeak could be used to teach orchestration/composition. And video editing.
Commitment:
My contributions would most likely be: I can test, I can document, and I can build end-user stuff.
Martin Kuball
Interests:
IMHO Squeak already has a lot of things we can use. But they need to be better integrated, polished up a little bit and bundled with some nice interface classes.
Commitment:
?
Russell Penny
Interests:
My main interest is making sure I can use any format I want in any of the tools that will be designed. Commitment:
I have been collecting format specification documents for a while now and am more than happy to help in that area.
Dean Swan
Interests:
1. Audio
A. MIDI
B. Synthesis
1) FM
2) Subtractive
3) Additive
4) Voice/Speech
C. Recording/Editing
1) Filtering (Simple and Adaptive)
2) Sample Rate Conversion
3) Effects (i.e. Reverb, Delay, Phasing, Flanging, Dynamics,
Pitch Modification, Time Stretching, etc.)
2. Video
A. Compression
1) Vector Quantization
B. Streaming
C. Editing
1) Colorspace Conversion
2) Scaling
3) Compositing
Commitment:
The majority of my experience is with audio DSP and anything related, so that's where I can probably contribute the most. Vector Quantization is something I have explored quite extensively on my own
Steve Elkins
Interests:
I'm more interested in (forming a better understanding of) Squeak's ability to integrate various media. I want to record stories and have her illustrate them, like narrated slide shows. I also want to set stories to music.
Commitment:
Testing and using for now.
Brian Vukmer
Commitment:
I'm interested in being a tester.
Yoshiki Ohshima
Interests:
Adding MPEG4 to Squeak Commitment:
My thought now is to write an equivalent of mp4player in Squeak, and call MPEG4IP library as plugin.
*Brad Fuller* *Interests:* Audio/Music *Commitment:* Help plan/code/test current audio base classes (need defn of what those are, which I can do). I will test, too.
No response (yet!) from:
Craig Latta
Mark McCahill
Juan Vuletich
Cornelius Huber
======================== We have 3 that have volunteered their time to test. Thanks! That is very important.
We don't quite have a quorum on what they would test, though ;-) But, I didn't think we would yet.
We also don't have a count on who would actually code.
First things first: from the replies, I see the following MM items of interest and at least one person is interested in writing/editing/enhancing code (and I'm sure I missed something, so please fire away and let's add it to the list): * Fix/enhance existing audio classes - could include the current soundrecorder/player tools * Add MPEG 4 player. * Dean has a lot of interests! It would help if you could prioritize. Will you help code? * Russell is looking at this from a MM usability/re-use POV (right?). In addition to OGG. * Martin feels that Squeak has a lot going for it already -- just needs cleaning/polishing and a new suit. Can you be more specific and what you feel needs cleaning up and what you would help with?
I certainly believe that people well feel compelled to help if they see a clear personal benefit. Finding the flag we can rally around is the difficult part -- especially since we are doing this for free and we are not working in the same timezones (however, round-the-clock programming can be productive and fun if organized right.)
Hey, we're getting closer. Looks darn good to me!
When I have more comments, I'll put out Part 3 (i know... i know... you just can't wait...)
brad
Brad,
I am seeing a lot of synergy actually. You do the audio tidy up, Blake does tools and doco, Dean does some very cool low level stuff which I sure forms the basis of a bunch of later work (i.e. better algorithms for OGG, etc than straight ports of "C"), Yoshiki, Martin and others do video, I handle file formats (with lots of help :) and the others test.
The main things I see happening from here that we need to resolve ASAP are a bunch of questions.
What do people see as the interfaces?
What have others seen in a multimedia framework? Is there a good one we can steal, I mean be influenced by.
What are the lowest common denominators?
What functionality does everyone want?
Just as an example what about these?
aURI playVideo
aURI playAudio
aURI play "uses a specific player based on the file type"
aURI edit "opens a specific editor based on the file type"
aURI editAudio "opens an audio editor on just the audio stream of the file"
When we figure out where we want to get to, we can work out how to get there.
As for the question below, yes I am interested in re-useability, something that everyone raves about for OO and Smalltalk but something that Squeak is very bad at doing in practice.
I am not a "programmer" although I program ;) so I will write code for the framework. I already have a simple MM framework but I need some help with the design, I am not experienced enough to know which are the better OO ways of doing things.
Russell
_____
From: squeak-dev-bounces@lists.squeakfoundation.org [mailto:squeak-dev-bounces@lists.squeakfoundation.org] On Behalf Of Brad Fuller Sent: Friday, 25 February 2005 12:52 PM To: The general-purpose Squeak developers list Subject: Re: Improving Squeak's Multimedia - Part 2 Followup
Brad Fuller wrote:
==== Attack Plan Finally, I propose that we decide on what to tackle and how to do it.
From a bird's eye view, we might look at 3 distinct phases (not necessarily
linear in order - can be approached simultaneously):
* Review/Discuss/Plan/Fix existing classes (which hopefully fix existing tools/apps) * Discuss-Design/Plan/Enhance existing classes * Discuss-Design/Plan/Add new capabilities: new classes, tools and end-user applications.
And, are you interested in helping?
As mentioned in my msg, I'm interested in mainly fixing and enhancing the audio base classes, first.
From the comments so far, we don't a clear winner -- not any particular
area that everyone feels compelled to work on. Nothing jumps out at me. Anything jump out at you?
Here's what I have so far:
Blake Watson
Interests:
I'd like to see an equalizer. I think it'd also be cool if Squeak could be used to teach orchestration/composition. And video editing.
Commitment:
My contributions would most likely be: I can test, I can document, and I can build end-user stuff.
Martin Kuball
Interests:
IMHO Squeak already has a lot of things we can use. But they need to be better integrated, polished up a little bit and bundled with some nice interface classes.
Commitment:
?
Russell Penny
Interests:
My main interest is making sure I can use any format I want in any of the tools that will be designed. Commitment:
I have been collecting format specification documents for a while now and am more than happy to help in that area.
Dean Swan
Interests:
1. Audio
A. MIDI
B. Synthesis
1) FM
2) Subtractive
3) Additive
4) Voice/Speech
C. Recording/Editing
1) Filtering (Simple and Adaptive)
2) Sample Rate Conversion
3) Effects (i.e. Reverb, Delay, Phasing, Flanging, Dynamics,
Pitch Modification, Time Stretching, etc.)
2. Video
A. Compression
1) Vector Quantization
B. Streaming
C. Editing
1) Colorspace Conversion
2) Scaling
3) Compositing
Commitment:
The majority of my experience is with audio DSP and anything related, so that's where I can probably contribute the most. Vector Quantization is something I have explored quite extensively on my own
Steve Elkins
Interests:
I'm more interested in (forming a better understanding of) Squeak's ability to integrate various media. I want to record stories and have her illustrate them, like narrated slide shows. I also want to set stories to music.
Commitment:
Testing and using for now.
Brian Vukmer
Commitment:
I'm interested in being a tester.
Yoshiki Ohshima
Interests:
Adding MPEG4 to Squeak Commitment:
My thought now is to write an equivalent of mp4player in Squeak, and call MPEG4IP library as plugin.
Brad Fuller Interests: Audio/Music Commitment: Help plan/code/test current audio base classes (need defn of what those are, which I can do). I will test, too.
No response (yet!) from:
Craig Latta
Mark McCahill
Juan Vuletich
Cornelius Huber
======================== We have 3 that have volunteered their time to test. Thanks! That is very important.
We don't quite have a quorum on what they would test, though ;-) But, I didn't think we would yet.
We also don't have a count on who would actually code.
First things first: from the replies, I see the following MM items of interest and at least one person is interested in writing/editing/enhancing code (and I'm sure I missed something, so please fire away and let's add it to the list): * Fix/enhance existing audio classes - could include the current soundrecorder/player tools * Add MPEG 4 player. * Dean has a lot of interests! It would help if you could prioritize. Will you help code? * Russell is looking at this from a MM usability/re-use POV (right?). In addition to OGG. * Martin feels that Squeak has a lot going for it already -- just needs cleaning/polishing and a new suit. Can you be more specific and what you feel needs cleaning up and what you would help with?
I certainly believe that people well feel compelled to help if they see a clear personal benefit. Finding the flag we can rally around is the difficult part -- especially since we are doing this for free and we are not working in the same timezones (however, round-the-clock programming can be productive and fun if organized right.)
Hey, we're getting closer. Looks darn good to me!
When I have more comments, I'll put out Part 3 (i know... i know... you just can't wait...)
brad
squeak-dev@lists.squeakfoundation.org
-
Blake -
Brad Fuller -
Brent Vukmer -
Cees de Groot -
goran.krampe＠bluefish.se -
Joshua Gargus -
Martin Kuball -
Russell Penney -
Steven Elkins -
Yoshiki Ohshima