Video Compression

Implementing effective video compression into your digital platforms requires knowledge of the most common codecs, digital formats, and optimization strategies.

What Is Video Compression?

Video compression is the process of reducing the file size of a video file by reducing the amount of data needed to represent its content, but without losing too much visual information. It's what powers every internet video streaming platform, including YouTube, Netflix, and TikTok. Video conferencing services like Facetime and Google Meet also compress video on the fly, with both maxing out videos to just 1080p.

Even the clearest, most high-resolution video you watch via the internet has been compressed, otherwise you might not be able to stream it.

There are two primary types of compression: lossy and lossless.

1. Lossy compression prioritizes space savings, meaning quality might take more of a hit in favor of reducing the file size as much as possible. The original and the compressed versions will have some differences in quality levels, but they're usually not enough to impact the viewing experience. It's ideal for cases where bandwidth is a concern, which is why most streaming platforms use it.

2. Lossless compression retains as much detail as possible, and you usually end up with larger file sizes. This type of compression is a popular choice for archival purposes and is less common in the context of video streaming.

Some of the benefits of using video compression include:

• Reducing storage costs: Companies that host video deal with overhead for storage, and compression helps them reduce that.
• Improving the user experience: Compressed video requires less bandwidth to transmit, giving users smooth playback even on slower connections.
• Increasing device compatibility: When video files are compressed, they're converted into a state that matches more device capabilities.

Without video compression, we couldn't have a world where 92% of all internet users watch videos online every month.

How Does Video Compression Work?

Video compression works by using a program called a codec to get rid of the redundant and/or imperceptible elements of a video. This is how it reduces the size of the file without affecting the quality of the footage. The difference between the original and compressed video's size is called the compression ratio.

There are two video compression algorithms that are in popular use today: intraframe and interframe compression.

1. Intraframe or spatial compression finds disposable elements in every video frame and compresses each one individually. It saves more detail from the original video and is ideal for footage with a lot of movement and visual changes.

2. Interframe or temporal compression works on a video chunk at once. It saves more space and is ideal for footage with little movement.

Modern video compression tools often leverage both of these algorithms to get the best results on each task.

Take a video where the camera is on a tripod and the speaker is seated in front of a full-wall mural. Since only the speaker is moving and the background isn't, it's possible to save only the elements that change and reuse the same version of the rest across multiple frames of video. This is how temporal compression gets rid of redundant details.

If two or more adjacent pixels represent nearly identical colors, it's possible to combine them into one color without it being detectable to the human eye. This process is called chroma subsampling, and it's an example of how compression gets rid of imperceptible details.

Other areas where video compression reduces details are in the file's frequency, dynamic range, and frame rate. For example, if a video was shot at 60 frames per second, compression could easily bring that down to 24 without a significant loss of quality.

After compression, delivery of video content on slow network connections becomes easier, through avenues like the Secure Reliable Transport (SRT) protocol.

What Are Some Common Video Streaming Compression Formats?

Video compression formats usually gain popularity if they're efficient, widely compatible, actively supported, and provided under a permissive (usually open-source) license. Some of the most popular formats for compressing video streams include:

• H.264 (MPEG-4 AVC): Short for Advanced Video Coding, it's the most common compression format today. Platforms like TikTok and YouTube use this format for efficient delivery, as it offers high-quality video at low file sizes.
• H.265 (HEVC): Short for High Efficiency Video Coding, it's the successor to H.264. It often produces better quality video at smaller file sizes than H.264, but at the cost of being more resource-hungry. It also requires a license for commercial use, which might explain its low adoption rates.
• VP8: Open-sourced by Google after its acquisition of inventor On2 Technologies, it's mainly used for short looping video formats, as an alternative to GIFs.
• VP9: Invented by Google, VP9 is a free and open-source competitor to H.265 and a successor to VP8. It's ideal for streaming above Full HD video, but is compatible with a relatively small variety of devices, so adoption is still slow.
• AV1: Short for AOMedia Video 1, it's meant to be a successor to VP9. Netflix is a major champion of this technology, recently rolling it out to its TV video player.
• FLV: Short for Flash Video, ‌made for use with Adobe Flash. Adobe stopped supporting Flash in December 2020 due to security flaws in the software.

What Are the Most Popular Video Compression Tools?

Hopefully, you've made your choice of video codec. The next step is choosing the tools you're going to use for compression.

1. APIs

Video compression APIs allow you to access powerful video compression algorithms from an external provider. They help you integrate video encoding, decoding, and compression functionality into your applications without having to build these features from scratch.

This explains why 54% of technology leaders say APIs speed up innovation and improve collaboration with their partners. More than six in 10 of them also say that APIs help their teams elevate their digital products and experiences.

The benefits of using APIs for video compression include:

• Time savings since you can implement new features faster
• Cost-efficiency because you're not shouldering the cost of building these features
• Increased functionality from access to more resources and features developed by a dedicated/specialized third party

However, some of the downsides of this approach include:

• Dependence on the vendor — if they stop providing the service, there's little you can do about it
• Limited customizability and control, as all you're doing via the API is accessing resources, not developing them
• Security concerns because you're pulling data into your systems from an external source

2. SDKs

Video compression SDKs give the tools you need to build compression into your platform yourself. These tools include code samples, documentation, libraries, and even APIs.

While SDKs give you more ownership as a developer, they often require more involvement than APIs to use.

Some of the pros of using an SDK for video compression include:

• Ownership over the video compression functionality when implemented
• Comprehensiveness, as they often include APIs in their toolkits

The potential downsides of using these tools include:

• Bloated code from tools that you don't need
• Limited flexibility because SDKs are often opinionated on how you should use them
• Steep learning curve compared to APIs

3. AI Video Compression Tools

AI can improve video compression through prediction-based compression, motion estimation and compensation, and quality assessment and optimization.

The team at YouTube has developed a program called MuZero, designed to optimize VP9 compression through reinforcement learning.

Some of the pros of using AI video compression tools include:

• Improved efficiency because these systems can improve their performance over time
• Higher adaptability, since they can handle unique compression tasks better than traditional tools

There are also a few downsides to these tools, including:

• Resource-intensive usage, as they require more computing power
• Less transparency due to proprietary algorithms

5 Best Practices for Streaming Video Compression

As a developer, understanding the best practices for video compression can significantly improve the quality and efficiency of your applications.

1. Choose the Right Format and Codec

In the section about video compression formats, we outlined how each one has some strengths and weaknesses. Choosing the right one requires you to evaluate your needs and how your choice will impact the user experience. For example, H.265 outperforms H.264 at higher resolutions, but it takes up more processing power. If your users are on weaker hardware and don't always seek out the highest quality, it might be better for you to go with the older technology.

2. Use Adaptive Streaming

The global median download speed on mobile devices is 33.97Mbps, and there are users on either side of that figure all over the world. Adaptive bitrate streaming allows you to compensate for differences in your end users' internet connections. When a user's connection is fast, they automatically get high-quality versions and vice versa. Look into protocols like HTTP Live Streaming (HLS) and MPEG Dynamic Adaptive Streaming Over HTTP (MPEG-DASH) to learn how you can incorporate adaptive streaming into your project.

3. Don't Ignore Audio Quality

Great video content isn't just about the visuals — the audio track is just as important. Over 90% of people value high-quality audio, so you have to make it part of your strategy for delivering a strong user experience.

Dealing with audio streams is also a balancing act between quality and file size, with lossless and lossy compression algorithms. Research digital audio compression techniques, codecs, and advanced algorithms for the right quality levels.

4. Test and Iterate

Run your platform through a series of stress tests to see how well its video compression functionality performs across a range of devices, browsers, and network conditions. For a more complete picture of its performance, involve your users by gathering feedback and running A/B tests. Use your findings to refine the video compression pipeline over time so it's more performant for as much of your user base as possible.

5. Keep up With Advancements in Video Compression

We've explored how the world of video compression formats is constantly evolving---the most successful platforms are the ones that have learned to adapt. To stay agile, you need to keep up with improvements in compression efficiency. Aside from keeping your platform relevant, this allows you to continue delivering a meaningful user experience. Adopting a more efficient video compression technology earlier than your competitors could be the advantage that gets you in front of more users.

Make Every Byte Count With Video Compression

Video compression in digital video, enabling efficient storage, transmission, and consumption of content across a wide range of devices. By leveraging the right algorithms and techniques, video compression helps reduce file sizes while maintaining acceptable visual quality. This makes it an indispensable tool for developers working on streaming services or video-as-a-service (VaaS) platforms. When selecting a video compression solution, you should consider efficiency, the developer experience, speed, supported formats, and integrations for seamless implementation.

Frequently Asked Questions