6 Things To Know About Video For The Web - Part 1

Video on the web is showing up in more and more places all the time. It's also easier to capture and with mobile decices, web cams and hand held cams. What we can see from this is that it's something that is here to stay. For those of us responsible for putting it up on the web (and even those who aren't!), there is a lot we need to know about what goes into creating and distributing video.

This post is meant to be a broad look at how far video has come and where it is today.  it provides some helpful tips about distributing video on the web.

Since blog posts that have numbers in their titles get more attention, I broke up video into six sections.

1. In what format should I present my video?

When viewing video on the web these days, there are 3 options: Quicktime(MOV), Silverlight(WMV), or Flash(FLV). More and more, you see video served up in the Flash player, but a few years ago you saw most video served using either Quicktime or a WMV player. And, before that, we saw the ever-buffering Real Media player. Why have things changed and why is video served mostly on the Flash player?

Availability

First, we need to look at how widely distributed each of these players is. If you look at the image below, you can see that the Flash player, when seen in the browser, has a significantly wider reach than the other players (note: results were conducted by a private institution and commissioned by Adobe).

Based on these stats, the Flash player just makes the most sense.

The Player

Next, we need to look at the players themselves. Within the browser you are not able to directly embed Flash video files -- but the opposite is true for Quicktime and WMV files, which have plugins built right into your browser. To deliver Flash video, you first need to have a custom player. Is this a problem? Yes and no. Yes, because you have to go through the hassle of coming up with your own player. No, because you have the incredible option of limitless customization to your player! The Quicktime player can be customized within itself but with only a handful of attributes. With the advent of Silverlight, a player for WMV files, players can now be completely customized.

When Flash was first capable of delivering video in 2002 (Flash Player 6), this posed a difficult problem because we did have easy access to the generic video player component that came with Flash -- but, like most generic components, it was clunky, large in file size, and not highly customizable. Today, it's significantly easier to add a custom player with options like YouTube. Vimeo will even encode video for you as will the highly popular standalone JW Player. A few years ago, I would have said the ease of the embedded Quicktime and WMV players was the way to go. However, since it's almost as easy to get your hands on an FLV player as using the existing QT and WMV players, any player here is a worthwhile option.

Quality of Video

The quality that a video player can put out is only as good as the quality of the video you put into it. Video on the web ranges from low-quality YouTube videos to those glorious 1080p movie trailers on Apple.com. The strength of a video player comes down to its codec. Quicktime and now Flash video can be delivered using the H.264 codec while Windows has its own proprietary codec. The difference between the two in the upper end of quality are not too far off. However, you need to keep in mind that while connection speeds have increased significantly over the years, internet video is still optimized for slower bandwidths and is, therefore, not delivered at its highest quality.

Now the question is, which codec delivers the best quality using the lowest bandwidth? Since all Quicktime video is delivered using the H.264 codec, which delivers video at a high quality but tends to be large in file size, you need to either wait for it to download or reduce the quality in video. Flash players' codecs called ON2VP6 introduced in Flash player 8 goes a long way in increasing the quality of video without increasing file size. It takes a toll CPU during playback more than the other codec, so it can be troublesome for older computers.

Mobile

The vast majority of mobile video is seen on the iPhone. The way the iPhone handles videos is by using an iPhone-specific tool developed by Apple and YouTube, which will allow you to watch YouTube-only videos. Video on the iPhone is not Flash video like it is on the real web but transferred using the Quicktime H.264 codec. Flash video may not be seen on the iPhone for some time due to the limitations to the iPhone itself, which isn't capable of running the Flash Player. Other devices like Smartphones and BlackBerries (or is it Blackberrys?) have had weak video capabilities and relied on custom programs to play .AVI and .MPEG files.

Flash video has been a no-show due to Flash's mobile player Flash Lite -- which they boast is on 800 million handheld devices but may be the most widely distributed piece of software on Earth that nobody uses -- and its inability to play video. That is until Flash Lite 3, which was released in late 2008, that finally enabled .FLV playback. Along with Flash Lite, Silverlight has thrown its hat in the ring with a Silverlight mobile player, which is currently being released on Smartphones along with Flash Lite.

In a word, video on mobile devices is spotty, but you can be assured that with the 300+ million mobile devices in the United States alone, things will surely change in the next few years. So when thinking about video for the mobile web, by this logic, the FLV format is the best if you intend to reach the most viewers.

Security

Most video distributers offer methods to protect your video from being copied and redistributed. WMV and Quicktime have adopted a form of Digital Restriction Management that gives the distributor control of how it want its files distributed (i.e. who can download, how many downloads, etc.) As of Flash Player 9 version 1, there was no standardized method for security or DRM in Flash video. None. So how are sites like NBC, Hulu and YouTube -- who use Flash Video -- protecting their video? For the people at YouTube, this is simple. They do not restrict their content. YouTube users upload video under the assumption that is it all public and can be freely distributed. We do see plenty of video of DRM content on YouTube (like parts of movies and tv shows, music videos, etc.). All I can say about that is there is a 1 billion dollar lawsuit between YouTube and Viacom currently playing out in the courts.

As of Flash player 9, version 2, there is a new format for Flash video with the extension .F4P, which now allows for protected video on the Flash player. It is for this reason that sites like NBC, who showed the 2008 Summer Olympics, and Netflix, who streams all movies in the WMV format, are using the Silverlight player.

2. How video on the internet is different from video on TV

Video on our TVs is not limited by download speeds like video on the web is, so TV can be delivered at a high and consistent quality. The way the video is delivered is as you would guess in keyframes (29.97 frames per second) where one frame follows another. The speed at which the frames are played fools our brains into believing that the images are actually moving.

Video on the web is different. If you've ever ripped a DVD onto your hard drive you've seen that the file is probably a gig or two. The way around this is by using what are called "difference frames." Video is still delivered in key frames; however, they are much fewer and farther between. Ever wondered what is going on when encoding video for the web? Well one of the many complex tasks is breaking out keyframes into these different frames. Encoders will start a typical video with a single keyframe then compare it with the next one. Take the two consecutive frames below.

Frame 1
Frame 2

The encoder will look at the first two keyframes and compare the difference in color at a pixel level. If it finds any colors that are exactly the same it removes them so you will get a frame like this (see below)

Difference Frame

What you are left with is about 10% of the original keyframe (note: the colors are actually inverted. Please pardon my lack of pshop skills). You can see how a movie that starts at 50 megs can be shrunk down to about 5 or so. The difference frames will vary in size with slower or quicker motion or as scenes change in color. Smart encoders will even pass through a video once looking for large changes in motion or color (for example, in Star Wars when the scenes changes from black space to a bright and sandy Tatouine) and just drop a fresh keyframe in the video. Using this method keyframe rates usually drop from 30 fps to about 300.

See Part 2 for the next 4 points about video on the web.