At 1:34 PM -0500 11/27/01, Mark Guzdial wrote:

Thanks for your enthusiasm, Mark!

Re:
>Why was it decided to invent a format rather than use the existing 
>Motion-JPEG standard (which I didn't know about until Bolot sent me 
>these URLs):
>
>http://bmrc.berkeley.edu/research/cmt/versions/4.0/doc/cmtmjpeg/MJPEG_ 
>chunkfile.html
>http://neptune.netcomp.monash.edu.au/cpe3013/MPEG/Reading/MJPEG/step1.htm
>
>The Motion-JPEG format sounds like essentially the same thing as the 
>movie-JPEG format, but I could be missing some of the subtleties.

Mostly because I didn't know about motion JPEG, either! (Well, I'd heard
the term, but knew nothing about it. Thanks for the links.) Motion-JPEG
seems fairly similar to JPEG movies, but with no provision for sound, as far
as I can see. How widely used is it? Do programs such as Quicktime deal
with it?

It certainly looks like it might be easy to write a converter to import
an MJPEG movie into a Squeak JPEG movie. If there is a way to include
soundtracks using arbitrary sound compression, then I'd have no objection
to just replacing the Squeak JPEG format with the MJPEG format. I don't
have time to do that myself, at least not in the near future.

Hmm, after studying these two web sites further, I'm less certain that Motion
JPEG could easily exploit with our current JPEG plugin. The key thing about
the Squeak JPEG movie format is that each frame is exactly a compressed
JPEG image, completely independent of all other frames, that can be sent
through the JPEG plugin. The links above suggest that perhaps Motion JPEG
frames are not in precisely this format, even though it uses the JPEG algorithm.
For example, individual frames might omit some header information that
is common to all movie frames. While we could probably extend the JPEG
plugin to handle such a format, the current JPEG movie uses exactly the
same data format as still images. Feel free to send any links that clarify
this issue...

Re:
>- Why is the soundtrack audio format Sun mulaw rather than something 
>more common like AIFF or WAV?

Soundtracks are in Sun audio file format. This format supports uncompressed
sample streams, as well as arbitrary codecs. I wanted  a format that could
be extended to use any new Squeak codecs that happened to come along. I
nearly created a new audio file format, but then realized that was silly. :->

Sun audio files are *really* simple--just 6 header words plus the data.


Re:
>- It looks like it's possible to do multiple soundtracks (though the 
>MPEG-to-Movie-JPEG example only create a single monophonic 
>soundtrack).  How does that work?  Are they mixed during playback, or 
>is there a way to choose between soundtracks during playback?

Multiple soundtracks haven't been implemented yet, either encoding or
playback, but I wanted to leave the option open for supporting them later.

Right now, you get the first sound channel from an MPEG movie that has
sound. If the movie has stereo sound, you get just the left channel. My thought
for later was to create one StreamingMonoSound for each channel and use
a mixer to mix them together. (Actually, tthat wouldn't quite work with the
current implementation of StreamingMonoSound.) The other approach would
be to make a StreamingMultichannelSound that mixes multiple sample
streams. Note that compression makes things a bit more complex, since
some of Squeak's codecs have provision for inter-mixed samples streams
but others (e.g. the GSM codec) do not. Of course, we can always create a
new format type for Sun audio files that does whatever kind of interleaving
makes sense.

Incidentally, the GSM codec does a pretty good job on the AlienSong.mpg
soundtrack, including a piano riff at the beginning. It's nowhere near as
clean as MP3, but it's acceptable, especially if you're listening through bad
laptop speakers anyhow. :->

The ADPCM codecs also work, but they don't handle random access properly.
It's on my list to fix that. For now, avoid them. (5-bit ADPCM sounds nearly
as good as mu-law, and it's only 5/8ths the bitrate.)


Re:
>I'm eager to get the plugin/VM to try it, particular on BUILDING some movies!

We're also thinking about how kids and teachers could import movies from
a digital video camera. It looks like it's reasonably easy with a firewire-equipped
Mac and iMovie. You'd also need to buy Quicktime Pro, which gives you the
ability to export a movie in a variety of formats, some of which could be read from
Squeak. One of these formats is simply to export a collection of individual frames
in PNG, JPEG, BMP, or various other formats. It's trivial to write a converter that
builds a movie from a collection of frames.

If anyone knows how one might do this on Windows, I'd like to know. The goal
is to avoid expensive software or video converter hardware. I think $25-$50 is
the most a kid should have to spend to get a movie into Squeak.

	-- John