At 1:34 PM -0500 11/27/01, Mark Guzdial wrote:
Thanks for your enthusiasm, Mark!
Re:
Why was it decided to invent a format rather than use the existing Motion-JPEG standard (which I didn't know about until Bolot sent me these URLs):
http://bmrc.berkeley.edu/research/cmt/versions/4.0/doc/cmtmjpeg/MJPEG_ chunkfile.html http://neptune.netcomp.monash.edu.au/cpe3013/MPEG/Reading/MJPEG/step1.htm
The Motion-JPEG format sounds like essentially the same thing as the movie-JPEG format, but I could be missing some of the subtleties.
Mostly because I didn't know about motion JPEG, either! (Well, I'd heard the term, but knew nothing about it. Thanks for the links.) Motion-JPEG seems fairly similar to JPEG movies, but with no provision for sound, as far as I can see. How widely used is it? Do programs such as Quicktime deal with it?
It certainly looks like it might be easy to write a converter to import an MJPEG movie into a Squeak JPEG movie. If there is a way to include soundtracks using arbitrary sound compression, then I'd have no objection to just replacing the Squeak JPEG format with the MJPEG format. I don't have time to do that myself, at least not in the near future.
Hmm, after studying these two web sites further, I'm less certain that Motion JPEG could easily exploit with our current JPEG plugin. The key thing about the Squeak JPEG movie format is that each frame is exactly a compressed JPEG image, completely independent of all other frames, that can be sent through the JPEG plugin. The links above suggest that perhaps Motion JPEG frames are not in precisely this format, even though it uses the JPEG algorithm. For example, individual frames might omit some header information that is common to all movie frames. While we could probably extend the JPEG plugin to handle such a format, the current JPEG movie uses exactly the same data format as still images. Feel free to send any links that clarify this issue...
Re:
- Why is the soundtrack audio format Sun mulaw rather than something
more common like AIFF or WAV?
Soundtracks are in Sun audio file format. This format supports uncompressed sample streams, as well as arbitrary codecs. I wanted a format that could be extended to use any new Squeak codecs that happened to come along. I nearly created a new audio file format, but then realized that was silly. :->
Sun audio files are *really* simple--just 6 header words plus the data.
Re:
- It looks like it's possible to do multiple soundtracks (though the
MPEG-to-Movie-JPEG example only create a single monophonic soundtrack). How does that work? Are they mixed during playback, or is there a way to choose between soundtracks during playback?
Multiple soundtracks haven't been implemented yet, either encoding or playback, but I wanted to leave the option open for supporting them later.
Right now, you get the first sound channel from an MPEG movie that has sound. If the movie has stereo sound, you get just the left channel. My thought for later was to create one StreamingMonoSound for each channel and use a mixer to mix them together. (Actually, tthat wouldn't quite work with the current implementation of StreamingMonoSound.) The other approach would be to make a StreamingMultichannelSound that mixes multiple sample streams. Note that compression makes things a bit more complex, since some of Squeak's codecs have provision for inter-mixed samples streams but others (e.g. the GSM codec) do not. Of course, we can always create a new format type for Sun audio files that does whatever kind of interleaving makes sense.
Incidentally, the GSM codec does a pretty good job on the AlienSong.mpg soundtrack, including a piano riff at the beginning. It's nowhere near as clean as MP3, but it's acceptable, especially if you're listening through bad laptop speakers anyhow. :->
The ADPCM codecs also work, but they don't handle random access properly. It's on my list to fix that. For now, avoid them. (5-bit ADPCM sounds nearly as good as mu-law, and it's only 5/8ths the bitrate.)
Re:
I'm eager to get the plugin/VM to try it, particular on BUILDING some movies!
We're also thinking about how kids and teachers could import movies from a digital video camera. It looks like it's reasonably easy with a firewire-equipped Mac and iMovie. You'd also need to buy Quicktime Pro, which gives you the ability to export a movie in a variety of formats, some of which could be read from Squeak. One of these formats is simply to export a collection of individual frames in PNG, JPEG, BMP, or various other formats. It's trivial to write a converter that builds a movie from a collection of frames.
If anyone knows how one might do this on Windows, I'd like to know. The goal is to avoid expensive software or video converter hardware. I think $25-$50 is the most a kid should have to spend to get a movie into Squeak.
-- John