High-Efficiency Video Coding (HEVC)

The emergence of High-Efficiency Video Coding with its H.265 compression standard marks a significant advancement in video streaming quality and visual content delivery.

What Is HEVC?

HEVC is a video codec (technology for encoding / decoding video) developed using the same video compression approach as AVC but with major updates. The High-Efficiency Video Coding (HEVC) standard was introduced in 2013 as an improvement on the prevailing H.264/MPEG-4 (AVC) standard. 

These advances reduce bitrate requirements while preserving visual clarity, allowing content providers to deliver high-quality streams with lower storage space and bandwidth.

HEVC's superior ability to deliver smooth video on mobile devices and livestream events has made it extremely popular.  It significantly reduces the choppiness that was common in the past due to frequent buffering on phones and laptops, enabling continuous streaming even with slower internet connections.  It also excels when high quality is required, allowing for real-time streaming at high resolutions up to 4K.

How Does HEVC Work? 

As with all video codecs, HEVC uses compression algorithms to reduce file sizes by removing redundant information, ideally, visual differences that will go unnoticed by viewers.

As the amount of compression increases, the number of unwanted visual distortions, such as blurriness and pixelation, also increases. At the end of the day, technical aims like file size reduction must be balanced with user experience concerns---the video has to pass the human eye's litmus test.

To understand HEVC compression, you first need to be familiar with the two existing categories of video compression: interframe and intraframe.

• Interframe compresses video using the differences between successive frames, which allows for more efficient streaming and storage.

• Intraframe compresses individual frames independently, which is helpful for making frame-by-frame adjustments in video editing and post-production.

The two can be used separately, but more complex technologies like HEVC generally use a combination of both, with software automatically detecting sections in videos with more or less movement.  

Intraframe Compression

Also called spatial compression, intraframe compression works with each video frame, processing them individually in a way similar to how still images are compressed into jpegs. It reduces redundant data within a single frame, such as small variations in color between adjacent pixels --- differences small enough to go unnoticed by the viewer.

Intraframe compression is ideal for high-movement videos because it addresses the subtle differences in each frame, ensuring that changes and movement are retained throughout the video.

Interframe Compression 

On the other hand, interframe compression looks for similarities between subsequent frames.  It's also known as temporal compression because it relies on similarities between frames over time.

Elements that are the same in neighboring frames can be reused without reloading, resulting in significantly reduced file sizes.

A great example would be a video of an interview with little change in the background. Interframe compression capitalizes on this lack of movement, saving the repeated parts of the image in an element called a keyframe (basically a snapshot of the screen at a given moment).  Using the data in this keyframe, only pixels that differ from it have to be updated in subsequent frames.

Both H.265 (HEVC) and its predecessor H.264 (AVC) use a combination of interframe and intraframe compression.

Key Features of HEVC

To fully understand why HEVC performs significantly better than AVC despite using a similar approach, we need to look at its key features in more detail.

• Improved Coding Efficiency: HEVC's primary coding efficiency improvement is due to its use of the coding tree unit (CTU) as its basic processing unit, versus the macroblock used in earlier methods.  The CTU allows the HEVC codec to process larger pixel blocks (64x64 pixels versus the previous standard of 16x16) resulting in much faster processing.

• Compression Ratio Improvements: The term compression ratio refers to the ratio between a video's file size uncompressed vs compressed. Compared to H.264, HEVC offers up to 50% better compression.  

• High-Resolution Support: HEVC supports high-resolution videos including 4K and 8K. Its ability to process large files is due largely to the use of the coding tree unit mentioned above, which saves time by processing larger blocks of each video frame.

• Improved Parallel Processing: HEVC divides each frame of a video into a grid of smaller rectangles, called tiles.  These individual tiles can be processed simultaneously thereby speeding up the processing time for compressing individual frames.  An advanced approach called Wavefront parallel processing (WPP) is sometimes used to optimize the technique even further.

• Video Coding Layer: The HEVC video coding layer refers to the hybrid interframe/intraframe approach discussed above. It includes more complex predictive algorithms used to determine which type of compression to use for particular segments of a video.  It uses the progressive scan method for generating the lines of each frame sequentially, as opposed to interlaced methods commonly used in the past.

• Color Spaces: A color space is a system for organizing and describing colors, either by digital or analog means.  A simple example of a digital color space is the 24-bit RGB model, which uses 8 bits (with values from 0-255) for each of three colors: red, green, and blue. By combining these three base colors with varying strengths, it creates a richer color palette. In the case of HEVC, it uses color space transforms (conversion to more efficient color space systems) to create higher quality videos with lower bitrates.

• Entropy Coding: Like its predecessor H.264, HEVC makes critical use of entropy coding in its efficient compression algorithms. Entropy coding is an advanced mathematical topic related to lossless data compression. HEVC exclusively uses a single advanced version of entropy coding called Context-Adaptive Binary Arithmetic Coding (CABAC), compared to AVC which uses multiple different algorithms.

• Intra Prediction: This method includes a handful of algorithms used together to predict spatial redundancies. It calculates predictions for new pixels in the same frame based on surrounding pixels.

• Motion Vector Prediction and Motion Compensation: The use of motion prediction models to predict movement between pixels in subsequent frames has been a part of interframe compression models even prior to the H.264. However, HEVC delivered improvements in motion prediction algorithms compared to all previous models.

• Range Extensions: HEVC's range extensions support additional color spaces and bit depths, extending the dynamic range and variety of colors available.

Advantages of HEVC 

The HEVC features noted above bring several advantages:

• Superior Handling of 4K/8K: HEVC's optimized compression algorithms are perfect for compressing high-resolution videos, leading to smaller file sizes and smoother playback than with AVC and other models. 

• Native HDR Support: HEVC supports High Dynamic Range (HDR) by default, using higher bit depth to provide a wider gamut of colors for a more vibrant and realistic viewing experience.

• Efficient bandwidth: The reduced file sizes of HEVC compression make it possible to deliver streaming video to those with slower internet connections.

• Smoother Mobile Playback: Despite the limited processing power, small storage size, and weak mobile data suffered on phones, HEVC still allows for smooth playback on mobile.

• Reduced Storage Requirements: HEVC's superior compression reduces the storage required for high-res video, benefiting video providers and consumers alike.

• Enhanced Streaming Quality Across Limited Network Connections: Even with less reliable network connections, HEVC delivers the smoothest possible video compared to its predecessors, with minimal buffering and glitches.

Disadvantages of HEVC 

HEVC isn't without its downsides though. Potential challenges and areas to consider include:

• Complexity: HEVC requires more processing power than its predecessors, due to its advanced computational complexity. 

• Compatibility: HEVC may not be compatible with all of your favorite devices and software, which could lead to playback restrictions for some users.

• Licensing Fees: HEVC is not free for commercial use and could therefore end up being more costly than other services.

• Encoding Time: For those who encode videos, such as providers, HEVC has a slower encoding time compared to some older codecs. Although this won't impact most viewers, encoding time might affect a provider's codec choice in certain instances.

• Browser Support and Livestream Challenges: not all browsers support live streaming of HEVC content, which may hinder its popularity.

Common HEVC Use Cases

The advantages of HEVC have made it a popular choice for many applications, including live streaming, mobile streaming, and video calling.  

Livestreaming 

The superior compression algorithms of HEVC make it an ideal option for live streaming, offering less glitching and delays than earlier codecs, with higher visual clarity. As a result, it is a favorite for sports broadcasting, social media, and other live events. 

4K Ultra Streaming

When it comes to HD, 4K video has nearly four times the pixels of normal 1080p HD video, making it a challenge to deliver in real time.  Thanks to HEVC's ability to deliver high-quality visuals at low bitrates, it excels at streaming this ultra-high-resolution video.

Video Calling

Online video conferencing and video calls are now an everyday part of work. Many professionals regularly connect via phone from a cafe, while others use complex systems for audio and video conferencing, focusing on high-definition.

When poor internet connection is an issue, HEVC can be a great solution for those needing to join a video call via mobile. On the other side of the spectrum, when high-quality video is paramount, HEVC is also great for the task.  

HEVC vs AVC

HEVC has many advantages over AVC, mostly related to its superior compression algorithms.  This results in file sizes as small as half the size of its predecessor's, which has a number of additional benefits. 

First, HEVC requires lower bandwidths for downloading and streaming, making it ideal for those with poor internet connections and providers who want to manage bandwidth from a business perspective.  

Second, it enables streaming of higher-quality videos compared to AVC but at the same bitrate. A side-by-side visual comparison shows that HEVC delivers much higher visual quality using the same internet connection and identical bitrate.  

Finally, HEVC is an excellent choice for high-resolution video, including 4K and even 8K, with smoother playback and overall superior performance. 

While the AVC codec is still a solid option for many general video compression needs, HEVC is now the preferred choice for the situations that matter most. It's the go-to codec for high-resolution video, low-bandwidth environments, mobile streaming, and efficient storage.

Frequently Asked Questions