This page describes the FFmpeg repository at a high level: its purpose, the libraries it provides, the command-line tools built from them, and how all components relate to each other. For detailed documentation of each subsystem, follow the links to the relevant wiki pages throughout this document.
FFmpeg is a cross-platform multimedia framework for decoding, encoding, transcoding, muxing, demuxing, streaming, filtering, and playing virtually any media format. The repository contains:
libav*, libsw*) providing reusable APIsffmpeg, ffprobe, ffplay) built on top of those librariesconfigure + Makefiles)This page is a map of the whole codebase. Deeper treatment of each area is in child pages — see page 2 for the build system, page 3 for the core libraries, page 4 for command-line tools, page 5 for codec implementations, page 6 for container formats, page 7 for hardware acceleration, page 8 for filters, page 9 for network protocols, and page 10 for testing.
The top-level directories map closely to library boundaries:
| Directory | Contents |
|---|---|
libavutil/ | Foundational utilities shared by all other libraries |
libavcodec/ | Codec (encoder/decoder) implementations and infrastructure |
libavformat/ | Container format muxers and demuxers |
libavfilter/ | Audio/video filter graph engine and filter implementations |
libswscale/ | Image scaling and pixel format conversion |
libswresample/ | Audio resampling and format conversion |
libavdevice/ | Input/output device abstractions (capture, display) |
fftools/ | Source for the ffmpeg, ffprobe, and ffplay binaries |
doc/ | Texinfo documentation and API change log |
tests/ | FATE test harness and reference data |
tools/ | Miscellaneous utilities (e.g., graph2dot) |
Sources: configure128-145 MAINTAINERS76-300
The libraries form a layered dependency graph. Lower layers have no upward dependencies.
Figure: Inter-library dependencies mapped to key public structs
Sources: libavutil/version.h81-93 libavcodec/version.h32-43 fftools/ffmpeg.h33-52
The foundational layer. Every other library depends on it.
Key public headers and their primary types:
| Header | Key types / functions |
|---|---|
frame.h | AVFrame, av_frame_alloc, av_frame_ref, av_frame_unref |
packet.h | AVPacket, av_packet_alloc, av_packet_ref |
buffer.h | AVBufferRef, av_buffer_alloc, av_buffer_ref |
hwcontext.h | AVHWDeviceContext, AVHWFramesContext |
pixfmt.h | AVPixelFormat, pixel format enumerations |
channel_layout.h | AVChannelLayout, av_channel_layout_* API |
opt.h | AVOption, av_opt_set, av_opt_get |
log.h | av_log, AVClass |
rational.h | AVRational, av_rescale_q |
Current version: 60.25.100 (LIBAVUTIL_VERSION_MAJOR = 60).
Sources: libavutil/version.h81-93 libavutil/frame.h1-50 libavutil/frame.c31-55
Provides all codec implementations and the encode/decode API.
Key types:
AVCodecContext — per-stream codec state, holds all encoding/decoding parametersAVCodec / FFCodec — public-facing and internal codec descriptorAVCodecParameters — codec parameters independent of a codec instanceAVCodecID — enumeration identifying every codec (libavcodec/codec_id.h)The send/receive API is the primary interface:
avcodec_send_packet → avcodec_receive_frameavcodec_send_frame → avcodec_receive_packetAll built-in codecs are registered through libavcodec/allcodecs.c.
Current version: 62.23.103.
Sources: libavcodec/version.h32-43 libavcodec/Makefile1-80 libavcodec/allcodecs.c1-30
Handles container format I/O — muxing and demuxing.
Key types:
AVFormatContext — top-level context for an open media file or streamAVStream — represents one elementary stream within a containerAVInputFormat — registered demuxer descriptorAVOutputFormat — registered muxer descriptorDemuxing entry point: avformat_open_input + avformat_find_stream_info + av_read_frame.
Muxing entry point: avformat_alloc_output_context2 + avformat_write_header + av_write_frame.
A directed-graph filter engine. Filters are connected into an AVFilterGraph. Data flows as AVFrame objects through AVFilterLink connections between AVFilterContext nodes.
Key types:
AVFilter — filter class descriptorAVFilterContext — instantiated filter within a graphAVFilterGraph — container for a complete filter graphAVFilterLink — connection between two filter padsbuffersrc, buffersink) bridge the graph to external codeSources: libavfilter/Makefile1-26 doc/filters.texi1-55
Scales video frames and converts between pixel formats. The primary context type is SwsContext. The modern API uses SwsGraph (a pipeline of SwsOp operations) and the sws_scale_frame function, while the legacy API uses sws_scale.
Resamples audio and converts between sample formats and channel layouts. Key type: SwrContext.
Three programs live under fftools/:
| Binary | Source entry point | Purpose |
|---|---|---|
ffmpeg | fftools/ffmpeg.c | Universal media converter / transcoder |
ffprobe | fftools/ffprobe.c | Media file analysis and metadata extraction |
ffplay | fftools/ffplay.c | Interactive media player |
All three share a common option-parsing infrastructure in fftools/cmdutils.c / fftools/cmdutils.h.
ffmpeg constructs a full transcoding pipeline at startup. Since FFmpeg 7.0, each stage runs in its own thread, orchestrated by the Scheduler (fftools/ffmpeg_sched.c).
Figure: ffmpeg transcoding pipeline and key source files
Key data structures in fftools/ffmpeg.h:
InputStream / InputFile — wraps a demuxer streamOutputStream / OutputFile — wraps a muxer streamDecoder / DecoderOpts — decoder configuration and stateFilterGraph / InputFilter / OutputFilter — filtergraph bindingOptionsContext — parsed command-line options for one input/output fileScheduler — thread and queue coordination across all pipeline stagesSources: fftools/ffmpeg.h33-630 fftools/ffmpeg.c1-50 fftools/ffmpeg_filter.c43-100 fftools/ffmpeg_sched.c1-40
The breadth of support is large. Representative categories, sourced from libavcodec/allcodecs.c and libavcodec/Makefile:
| Category | Examples |
|---|---|
| MPEG family | MPEG-1, MPEG-2, MPEG-4, H.263 |
| H.26x / AVC | H.264 (native + hw), HEVC (native + hw), VVC/H.266 |
| AV1 | Native decoder, libaom / libdav1d / librav1e / SVT-AV1 |
| Lossless | FFV1, HuffYUV, Lagarith, PNG, APNG |
| Image | JPEG, MJPEG, JPEG-LS, JPEG 2000, JPEG XL, WebP, BMP, TIFF |
| Legacy game/animation | Bink, Smacker, SANM, Cinepak, Indeo |
| Professional | DNxHD/DNxHR, ProRes, CineForm (CFHD), DPX |
| Category | Examples |
|---|---|
| Lossy | AAC, USAC, MP3, AC-3, E-AC-3, Opus, Vorbis, DTS |
| Lossless | FLAC, ALAC, WavPack, APE, TTA |
| Speech | AMR-NB, AMR-WB, G.729, G.711, Speex |
| ADPCM family | IMA ADPCM, MS ADPCM, DK3/DK4, EA, Westwood, ~30 variants |
| PCM | Signed/unsigned 8–32 bit, float, A-law, µ-law, Bluray PCM |
| Category | Demuxer | Muxer |
|---|---|---|
| ISOBMFF (MP4/MOV) | ✓ | ✓ |
| Matroska / WebM | ✓ | ✓ |
| MPEG-TS | ✓ | ✓ |
| AVI | ✓ | ✓ |
| FLV / Enhanced FLV | ✓ | ✓ |
| HLS | ✓ | ✓ |
| MPEG-PS / VOB | ✓ | ✓ |
| OGG | ✓ | ✓ |
| MXF | ✓ | ✓ |
| IAMF | ✓ | ✓ |
Sources: libavcodec/allcodecs.c1-50 libavcodec/Makefile190-500 Changelog1-170
FFmpeg supports GPU-accelerated decoding and encoding through multiple backends. The hardware context abstraction lives in libavutil/hwcontext.h (AVHWDeviceContext, AVHWFramesContext).
| Backend | Platform | API type |
|---|---|---|
| Vulkan | Cross-platform | Decode, encode, filter |
| VAAPI | Linux (Intel/AMD) | Decode, encode |
| NVDEC / NVENC | NVIDIA | Decode, encode |
| CUVID | NVIDIA | Decode |
| D3D11VA / D3D12VA | Windows | Decode, encode |
| VideoToolbox | macOS/iOS | Decode, encode |
| MediaCodec | Android | Decode, encode |
| QSV (Intel) | Cross-platform | Decode, encode |
| VDPAU | Linux (NVIDIA) | Decode |
The get_format callback on AVCodecContext is the primary hook that selects hardware pixel formats at decode time.
Sources: configure351-375 Changelog59-65
The build system is driven by the top-level configure shell script and a hierarchy of Makefiles.
configure probes the toolchain and system, then generates config.h and per-library Makefiles. Components are conditionally compiled using OBJS-$(CONFIG_FEATURE_NAME) Makefile variables.
Option categories accepted by configure:
| Category | Examples |
|---|---|
| Licensing | --enable-gpl, --enable-nonfree, --enable-version3 |
| Programs | --disable-ffmpeg, --disable-ffplay, --disable-ffprobe |
| Components | --enable-decoder=NAME, --disable-encoder=NAME, --disable-filters |
| External libs | --enable-libx264, --enable-libopus, --enable-libvpx |
| Hardware | --enable-cuda-nvcc, --disable-vaapi, --disable-vulkan |
| Toolchain | --arch=, --cc=, --cross-prefix= |
| Optimization | --disable-asm, --disable-avx512, --disable-neon |
For full detail on the build system, see page 2.
Sources: configure59-543 libavcodec/Makefile1-90 libavfilter/Makefile1-30
Each library follows a MAJOR.MINOR.MICRO scheme (similar to Semantic Versioning), defined in each library's version.h. The MICRO component starts at 100 rather than 0.
| Library | Current version | Identifier prefix |
|---|---|---|
| libavutil | 60.25.100 | lavu |
| libavcodec | 62.23.103 | lavc |
| libavformat | (see avformat/version.h) | lavf |
| libavfilter | (see avfilter/version.h) | lavfi |
| libswscale | (see swscale/version.h) | lsws |
| libswresample | (see swresample/version.h) | swr |
All public API additions and removals are logged chronologically in doc/APIchanges.
For detailed versioning policy and ABI compatibility guarantees, see page 11.1.
Sources: libavutil/version.h56-122 libavcodec/version.h19-45 doc/APIchanges1-50
Several types are used pervasively across library boundaries:
Figure: Central data flow types and which libraries own them
Sources: libavutil/frame.h1-100 fftools/ffmpeg.h467-514 libavcodec/Makefile30-65
Maintainership is tracked in MAINTAINERS by subsystem, with entries classified [0] (no current maintainer), [1] (light-touch), or [2] (actively maintained). Automated reviewer suggestions for pull requests are configured via .forgejo/CODEOWNERS, which maps path patterns to GitHub usernames using Go-style regular expressions.
Licensing is LGPL 2.1+ by default. Enabling --enable-gpl or --enable-nonfree during configuration widens what components are compiled in, but changes the license of the resulting binaries.
Sources: MAINTAINERS1-75 .forgejo/CODEOWNERS1-30 configure97-103
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.