INFO-I400 Topics in Informatics

Graphics, Animation, and Multimedia for the Web

Lesson 16: Audio and Video

Prerequisites:

Learning Objectives

Reading Assignment

Read both of these.

Pilgrim et al. do an excellent job of explaining container formats, codecs, and patent licensing issues, and converters, so I’m not going to repeat it. You need to read the assignment.

Both Pilgrim and the Mozilla reading explain how to use the video and source elements. The Mozilla reading also explains the audio element, and gives some information about scripting both audio and video. Again, read the assignments.

Patent Issues

If you use (encode, distribute, decode) a patent-encumbered format, such as the various MPEG formats (including MP3 audio format), you may be responsible for paying royalties!

The MP4 licensing is totally incomprehensible to me. Does it make sense to you? Do we have to pay royalties to build and sell encoders/decoders? to use encoders/decoders? to distributed encoded media? So if we buy an encoder, encode some video with it, and then distribute the encoded video, we have to pay twice? three times? Really?

Should software be patentable? Should algorithms be patentable? Should implementation of algorithms in software be patentable?

Audio Element

Basic usage:

<audio id="audio0" src="AUDIO-URL" controls>
  FALLBACK CONTENT
</audio>

The id allows your JavaScript to reference the element easily; src is the location of the audio file; controls causes the built-in player to show controls, so the user can start and pause the playback. FALLBACK CONTENT is shown only if the browser does not recognize the audio element; this allows you to show a message or provide an alternative viewing mechanism (such as Flash, a Java applet, or a link to download the audio file).

Video Element

Basic usage:

<video id="video0" src="VIDEO-URL" width="WIDTH" height="HEIGHT" controls>
  FALLBACK CONTENT
</video>

The id, src, controls, and FALLBACK CONTENT are like those for the audio element. If you know your video’s dimensions, you should specify width and height (in pixels); this allows the web browser to allocate the right amount of space on the page even before the video is loaded, reducing the “jumpiness” of the web page as it loads up. Also, you can make it larger or smaller than the actual resolution of the video, but the browser will, I think, preserve the aspect ratio (width/height), even if you do not.

If you want to provide multiple codecs (and possibly owe royalties for their use), then use this form as described by Mark Pilgrim:

<video id="video0" width="WIDTH" height="HEIGHT" controls>
  <source src="VIDEO-URL" 
          type="MIMETYPE; codecs=VIDEOCODECSPEC, AUDIOCODEC"/>
  FALLBACK CONTENT
</video>

Additional FALLBACK CONTENT can include Flash or Java applets, and/or fallbacks such as Kaltura.

Example 16-01

Basic audio and video elements.

General Advice

Avoid the autoplay attribute; it is annoying!

Scripting the Audio or Video Player

// Get the audio or video element
var player = document.getElementById("audio0");
// or 
var player = document.getElementById("video0");

// Start or stop playback
player.play();
player.pause();

// Change the source file in the same player
player.src = "ALT-MEDIA-URL";

// Hide and show the player
player.hidden = true;
player.hidden = false;

// Change the size of the player
player.width = 300;
player.height = 400;

Media events include "ended" (when playback completes) and "canplaythrough" (when enough of the file has loaded that you can hope to play it through to the end). For a complete list, see Media events.

Example 16-02

Audio scripting. Use a hidden audio element to play different sounds in response to button clicks.

Summary of Audio, Video, and Source Elements

Problems

How could you script an audio or video element to play a series of media files, one after another?

How could you script an audio element to play various sounds, at different time offsets from a single audio file, in response to “events” in a game?

Possible Discussion Assignment

What reasons do Microsoft and Apple give for not supporting WebM and Ogg Vorbis in their browsers? Are their reasons valid? Why or why not?

Credits

Reference