Choosing video codecs
What is a video?
Before we jump into codecs, let’s start from the beginning and look at what a video is. Thanks to the poor eyesight of humans, we can create an illusion of moving images by displaying several still images in quick succession. Think of a flipbook like the one below. You make a lot of pictures that are slightly different and quickly show them after one another. Voila, video magic.
Credits to TraceLoops
What is a codec?
Now that we settled on what a video is. Let’s move on to how it is stored. You could save every image of the video, also called a frame, and you would have a watchable video when making a flipbook like the one we have just seen. It is lots of work to completely redraw every frame when most of it is the same. Enter the codec, which is a way to store video more efficiently. A codec uses the fact that frames have many similarities between them and only storing the parts that change between frames. This description is, of course, a vast simplification but illustrates the basic idea behind codecs.
More formally, a codec describes how to encode a video to store it more efficiently. Then how to then decode it again, to be able to play it back to a viewer.
There are many different codecs, and we are going to have a look at a few of them — both from a brief historical and technical perspective.
Advanced Video Coding (AVC) or H264
If you had to choose only one codec to use, this is the one you should select. It has pretty much everything you would ever want from a codec. Works everywhere, as much as something can work everywhere on the internet, at least. Meaning that pretty much any device you will encounter today can decode and play a video encoded with H264.
HEVC or H265
Just as it sounds, this is a sequel to H264. Not only is it harder to pronounce, but it also has about 40% lower bitrate for the same quality. It takes longer than H264 to encode, but that is all right in most situations. There are a few other problems with it, however.
Most of them stem from the fact that it has now been more than since the codec was finalized, and it is still unclear what kind of royalties you need to pay.
There are no fewer than three patent pools that we know about, and at least one of them hasn’t stated their licensing terms yet. The only way to describe it is that it is a mess. That probably explains the other problem with HEVC. The low adoption considering that it, from a technical perspective, is a capable codec. There is only support in Safari, Edge, and IE browsers. Support on iPhone but not Android.
VP9
VP9 is a codec released by Google that has a similar performance as HEVC. It said to be royalty-free. Even though there have been challenges to its royalty-free status, nothing significant has yet happened to change that. The compatibility is, interestingly enough, almost the inverse of HEVC. With support on all major browsers except Safari and IE. Support on Android but not iPhone.
AV1
Before talking about AV1, it is necessary to mention the Alliance for open media. The Alliance consists of several encoding initiatives. The HEVC licensing terms, or more lack of terms, could have been what sparked its creation.
AV1 is an open and stated royalty-free codec. The compression is better than both VP9 and HEVC. It is still a very new codec, and therefore device supports still has a long way to go.
Choosing codecs
Which codecs should you use when encoding video. Unfortunately, the only reliable answer is that it depends.
Luckily, there is an easy rule you can follow. Unless you already know which codec you need or want to use. You should almost certainly use H264. It is the standard that works on pretty much any device you’ll encounter. None of the other codecs we have looked at here has the same reach.
Support table
Below is an attempt at putting all the data on different codecs together. To be able to see the strengths and weaknesses of codecs side by side.
The Future
VVC or H266
If you think this is the successor to H265, you are right. It is still a work in progress but looks interesting so far. If they can avoid making the same license mess as with HEVC, this could really be something.
What to keep an eye on
AV1, whatever happens, it is going to be interesting. Unless the royalties mess around HEVC is somehow magically resolved, it will probably never gain enough traction to be all that interesting, which is a bit of shame given its performance.
VVC, to see what is going to happen with the licensing.