<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><o:SmartTagType
namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PersonName"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";
color:black;}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
pre
{margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Arial;
color:blue;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
@page Section1
{size:612.0pt 792.0pt;
margin:72.0pt 90.0pt 72.0pt 90.0pt;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor=white lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>Brad,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'> I understand your interest
and frustration with sound but I think we need to go further than just audio
and video. Why? Because a lot of formats are reused by various audio, video and
still image formats. If we come up with a framework which fits all these in,
then we don’t have to keep reinventing or hacking existing classes to
make them do what we want. <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>I realize I am being vague so here is an
example:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>OGG files are made up of a container (OGG
format) which usually contains one or more Vorbis streams. However there is a
small push to use OGM containers which are OGG containers with AVI video and Vorbis
audio streams. The metadata of OGG or OGM files can have JPEG files in them.
JPEG has metadata which can be EXIF data showing camera information/who took
the photo/etc. AVI streams can be based on MPEG which is based on JPEG.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>So you can see that once we have a basic
framework, we can re-use classes to fully support the variations that seem to
pop-up all the time.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>Now I see 2 parts to this, the “sinks”
that play sound/video and the ability to read, write and transform various
formats into the native Squeak ones. Then the tools you talk about sit on top
of those fundamental classes.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>Now one of my biggest concerns is that we
should try to develop EVERYTHING in Squeak with as few plugins as possible. I
wrote the OGG decoder in “pure” Smalltalk so it is possible. Of
course by converting a small part of the decoder to a plugin I sped everything
up 10 fold but that came later ;) <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>I think this is important to remove
platform dependencies or dependencies on external libraries. We may end up
using them purely for performance but I think we should have a goal of not using
them in the first part.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>Obviously the “sinks” would
have to be plugins as they are very platform specific.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'>Russell<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=blue face=Arial><span style='font-size:
12.0pt;font-family:Arial;color:blue'><o:p> </o:p></span></font></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
color=black face="Times New Roman"><span style='font-size:12.0pt;color:windowtext'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 color=black face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;color:windowtext;font-weight:bold'>From:</span></font></b><font
size=2 color=black face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma;
color:windowtext'> squeak-dev-bounces@lists.squeakfoundation.org
[mailto:squeak-dev-bounces@lists.squeakfoundation.org] <b><span
style='font-weight:bold'>On Behalf Of </span></b>Brad Fuller<br>
<b><span style='font-weight:bold'>Sent:</span></b> Sunday, 20 February 2005
5:51 AM<br>
<b><span style='font-weight:bold'>To:</span></b> <st1:PersonName w:st="on">The
general-purpose Squeak developers list</st1:PersonName><br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: Improving Squeak's
Mutlimedia</span></font><font color=black><span style='color:windowtext'><o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'>Thanks for the reply and interest, Martin. See below<br>
<br>
Martin Kuball wrote: <o:p></o:p></span></font></p>
<pre wrap=""><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'>Am Friday 18 February 2005 21:09 schrieb Brad Fuller:<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt' type=cite><pre wrap=""><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Is anyone interested in improving the multimedia facilities in<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Squeak?<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'><snip><o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre></blockquote>
<pre wrap=""><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'>I'm definitely interested. Unfortunately I'm not very good in writing <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>long Mails. I will try to sum up my plans and ideas in the next <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>couple of days. But here are some remarks that come to my mind when <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>reading your EMail.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>What is Multimedia? Certainly it includes Audio and Video. But I think <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>there is more.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'>Yeah, I didn't really explain, kinda vague.<br>
I'm referring to only realtime full-motion video and sound/music only. I wasn't
refering to static graphics, text, etc. The rest seems to be addressed
well by others. To me, both audio and video are in need of more attention.<br>
<br>
<o:p></o:p></span></font></p>
<pre wrap=""><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>There are different levels or types of multimedia software. One type <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>is concerned with the creation of new content. Another type deals <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>with the manipulation of existing content. I think my AudioVideoLib <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>ist an example for the latter type. Playback another.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'>Right. I think we could view this in layers:<br>
* Low-level: Most of what is in the VM that addresses the platform-specific.<br>
* Mid-level: Fundamental classes that connect to the VM and provide fundamental
<br>
facilities for upper level "apps". This could probably
be divided into two.<br>
* Upper-level: Applications for users - creation of content (like vegas, premiere,
<br>
sound forge, protools) playback of content (like windows media
player, xmms) that make<br>
use of the fundamental classes.<br>
<br>
(Perhaps it is not as clean a line as described here. But, if I remember
correctly, the sound recorder and sound playback are intertwined a bit and
several things are hard-coded (like sample-rate) that would be better as a
setting for the user. The fundamental recording/playback should be separate
from the user audio application. A while back, I've pulled these apart and also
made the AIF/WAV file access classes more general. But, it was more for just my
use and would like to make it more general for the image. Don't hold me to this
explanatin as the current state as I'm writing this from memory of 6 months ago
-- written here purely for explanation.)<br>
<br>
Approaching the problems could be addressed in layers, as well. Perhaps there
are volunteers that would like to work on the VM some that work on the
fundamental classes, etc.<br>
<br>
<br>
<o:p></o:p></span></font></p>
<pre wrap=""><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>At the moment I'm trying to finish some missing parts of the AVI stuff <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>and make everything more robust. Another important point is to really <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>put it to use and find out the limits of what you can do with it <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>(speed wise I mean) and to improve the API.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'>that could be helpful for many.<br>
<br>
<o:p></o:p></span></font></p>
<pre wrap=""><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>By the way I'm really excited about the things going on here at the <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>moment. I hope we can move squeak to a new level in the next few <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>month.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Martin<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre></div>
</div>
</body>
</html>