FFmpeg in Production: Codecs, Performance, and Licensing

If you've built a product that handles video uploads or live streams, you've probably encountered FFmpeg. Once you're in production, you need to decide which codec plays on which devices, how much CPU time you're burning per video, and sometimes whether you need a lawyer to understand patent licensing.

What FFmpeg is

FFmpeg describes itself as a "complete, cross-platform solution to record, convert and stream audio and video" on the official about page.

Developers use two main parts:

• CLI tools - ffmpeg for transforming media and ffprobe for inspecting streams and metadata
• Libraries - libavcodec, libavformat, libavfilter for embedding media pipelines into applications

FFmpeg handles almost any format, which makes it ideal for ingest pipelines where you don't control what users upload.

Your first encoding command

Here's a standard H.264 encode with AAC audio:

1
ffmpeg -i input.mp4 -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k output.mp4

Breaking this down:

• -i input.mp4 - source file
• -c:v libx264 - use H.264 video codec
• -preset medium - encoding speed/efficiency balance
• -crf 23 - quality target (lower = better quality, bigger file)
• -c:a aac - use AAC audio codec
• -b:a 128k - audio bitrate

Before encoding anything, inspect it:

1
ffprobe -hide_banner -i input.mp4

This shows you what streams exist, what codecs they use, and what metadata is present. You need this information to make encoding decisions.

Containers vs codecs

A container (MP4, MKV, WebM) is the wrapper. A codec (H.264, VP9, AV1) is the actual compressed video data inside. You can have a perfectly valid MP4 file that won't play anywhere because the codec inside is unsupported.

Mozilla's media containers guide explains how containers and codecs fit together.

For browser-specific constraints, see MDN's video codec guide.

Troubleshooting container/codec mismatches

When files won't play or convert:

1. Check streams: ffprobe -hide_banner -i file
2. Test decode: ffmpeg -v error -i file -f null -
3. Verify codec support in target container
4. Check browser compatibility at caniuse.com

Codec tradeoffs

Codec choice means balancing compatibility, compression, encoding cost, and licensing. FFmpeg's AV1 guide notes that AV1 typically delivers ~30% better compression than VP9 and ~50% better than H.264 in relevant scenarios.

Encoding speed comparison (1080p30 content)

These storage savings compound with the encode time costs. AV1 saves storage but burns significantly more compute.

Rate control: CRF vs target bitrate

FFmpeg's H.264 guide recommends two approaches: CRF (quality-based) and two-pass ABR (bitrate-targeted).

"For most use cases, CRF is the recommended rate control mode. It provides the best quality per bit, and unlike target bitrate encoding, every frame gets the bits it deserves." — Handbrake video encoding guide

CRF example (quality-based):

1
ffmpeg -i input.mp4 -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k output.mp4

Target bitrate example:

1
ffmpeg -i input.mp4 -c:v libx264 -b:v 2500k -c:a aac -b:a 128k output.mp4

For deeper understanding of CRF, see slhck's CRF guide.

Hardware acceleration

FFmpeg's hardware acceleration guide covers the different backends.

"Hardware encoders trade some quality for speed. For user-generated content where encode time matters more than squeezing out the last bit of efficiency, hardware acceleration is often the right call." — FFmpeg Hardware Acceleration Guide

CPU baseline:

1
ffmpeg -i input.mp4 -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k output.mp4

NVENC version:

1
ffmpeg -hwaccel cuda -i input.mp4 -c:v h264_nvenc -preset medium -cq 23 -c:a aac -b:a 128k output.mp4

Hardware encoders deliver lower quality at the same bitrate compared to software encoders, but the throughput difference is significant.

Throughput example: 1080p → 720p H.264 transcoding

Media economics

Cloudflare's video delivery overview explains why video is economically different from typical web assets.

Most platforms generate multiple renditions:

Storage impact (60-minute 1080p video)

For a platform with 100,000 hours of content, AV1 vs H.264 could mean ~$2,300/month in storage savings. But remember: AV1 encoding is 10-50x slower than H.264.

Licensing

FFmpeg licensing is documented on the project's legal page and its LICENSE doc. Codec patent licensing is separate.

H.264 and H.265 have patent pools (examples: Via LA AVC/H.264, Access Advance HEVC). The licensing terms vary based on distribution model, revenue, and geography.

AV1 is positioned as royalty-free by the Alliance for Open Media. VP9 is generally treated as royalty-free in most contexts, though you should verify for your specific use case.

When to worry about licensing: If you're distributing video players, selling encoded content, or operating at significant scale, consult legal counsel. Patent licensing terms change and vary by jurisdiction.

FAQs

These FAQs cover the most common questions developers ask when moving FFmpeg from experimentation to production.

1. What is FFmpeg?
   FFmpeg is a toolkit for recording, converting, filtering, and streaming audio/video, described on the official site as a "complete, cross-platform solution to record, convert and stream audio and video" (FFmpeg About).

2. How to use FFmpeg to convert MKV to MP4
   If streams are compatible, stream copy:

   bash

   1
   ffmpeg -i input.mkv -c copy output.mp4

   If needed, map all streams:

   bash

   1
   ffmpeg -i input.mkv -map 0 -c copy output.mp4

   If it fails, it's usually container/codec mismatch—see MDN containers.

3. How to convert WebM to MP4 using FFmpeg
   Inspect:

   bash

   1
   ffprobe -hide_banner -i input.webm

   Encode:

   bash

   1
   ffmpeg -i input.webm -c:v libx264 -preset medium -crf 23 -c:a aac -b:a 128k output.mp4

   References: FFmpeg Encode/H.264, MDN codecs.

4. What is CRF in FFmpeg?
   CRF is quality-based rate control recommended in FFmpeg Encode/H.264. Deeper explanation: slhck CRF guide.

5. How to set bitrate in FFmpeg

   bash

   1
   ffmpeg -i input.mp4 -c:v libx264 -b:v 2500k -c:a aac -b:a 128k output.mp4

   See also two-pass ABR in FFmpeg Encode/H.264.

6. FFmpeg default audio codec when converting to MP4
   Check:

   bash

   1
   ffmpeg -h muxer=mp4

   Reference discussion: Stack Overflow muxer defaults. AAC notes: FFmpeg Encode/AAC.

7. Why doesn't FFmpeg support feature [xyz]?
   See the official FFmpeg FAQ.

8. I cannot read this file although this format seems to be supported by FFmpeg
   This is commonly container/codec mismatch—see the FFmpeg FAQ. Debug:

   bash

   1
   2
   ffprobe -hide_banner -i input_file
   ffmpeg -v error -i input_file -f null -

9. Are there examples illustrating how to use the FFmpeg libraries?
   Start with Using libav* and the ffmpeg-codecs docs.