This page documents how vLLM's Python package dependencies are declared, organized, tracked, and installed. It covers the requirements/ directory structure, version pinning for Docker builds, the generate_versions_json.py tool, and the use_existing_torch.py helper for nightly PyTorch builds.
For how dependencies are consumed during Docker image construction, see 11.3. For the CMake build system and C extension dependencies, see 11.2. For ROCm-specific build setup, see 11.5.
All Python dependency declarations live in the requirements/ directory. The files split dependencies by purpose (runtime, build, test) and by platform (CUDA, ROCm).
| File | Purpose | Locked? |
|---|---|---|
requirements/common.txt | Core runtime dependencies, all platforms | No |
requirements/cuda.txt | CUDA runtime deps; includes common.txt | No |
requirements/rocm.txt | ROCm runtime deps; includes common.txt | No |
requirements/build.txt | Build-time deps (cmake, setuptools, torch) | No |
requirements/rocm-build.txt | ROCm build deps; includes common.txt | No |
requirements/test.in | Test dep declarations (human-edited) | No |
requirements/test.txt | Fully-locked test deps (auto-generated) | Yes |
requirements/rocm-test.txt | ROCm test deps; includes common.txt | No |
requirements/nightly_torch_test.txt | Test deps for nightly PyTorch builds | No |
requirements/dev.txt | Development meta-requirements | No |
requirements/lint.txt | Linting tools | No |
requirements/kv_connectors.txt | Optional KV connector packages | No |
Sources: requirements/common.txt requirements/cuda.txt requirements/rocm.txt requirements/build.txt requirements/test.in requirements/test.txt1-5 requirements/rocm-test.txt requirements/rocm-build.txt requirements/nightly_torch_test.txt
The following diagram shows the include relationships between requirements files and how they map to the files on disk.
Requirements File Include Graph
Sources: requirements/cuda.txt1-2 requirements/rocm.txt1-2 requirements/rocm-build.txt1-2 requirements/rocm-test.txt1-2 requirements/test.txt1-3 requirements/build.txt1 pyproject.toml1-14
common.txt)requirements/common.txt lists packages required on every platform. Notable categories:
| Category | Key Packages |
|---|---|
| Tokenization | transformers>=4.56.0,<5, tokenizers>=0.21.1, sentencepiece, tiktoken>=0.6.0 |
| Serving | fastapi[standard]>=0.115.0, openai>=1.99.1, aiohttp>=3.13.3, pydantic>=2.12.0 |
| Observability | prometheus_client>=0.18.0, prometheus-fastapi-instrumentator>=7.0.0, opentelemetry-sdk>=1.27.0 |
| Structured output | xgrammar==0.1.29, outlines_core==0.2.11, lm-format-enforcer==0.11.3, llguidance>=1.3.0 |
| Quantization | compressed-tensors==0.13.0, gguf>=0.17.0 |
| Communication | pyzmq>=25.0.0, msgspec, grpcio, grpcio-reflection |
| Multimodal | pillow, mistral_common[image]>=1.9.1, opencv-python-headless>=4.13.0 |
| Build tooling | ninja, setuptools>=77.0.3,<81.0.0 |
Version ranges in common.txt are intentionally loose to remain compatible across platforms. Exact pins are enforced only in locked files.
Sources: requirements/common.txt
cuda.txt)requirements/cuda.txt adds the CUDA-specific stack on top of common.txt:
-r common.txt
numba == 0.61.2
ray[cgraph]>=2.48.0
torch==2.10.0
torchaudio==2.10.0
torchvision==0.25.0
flashinfer-python==0.6.4
nvidia-cutlass-dsl>=4.4.0.dev1
quack-kernels>=0.2.7
PyTorch is pinned to an exact version (torch==2.10.0). The corresponding CUDA wheel index is passed at install time (e.g., --extra-index-url https://download.pytorch.org/whl/cu129).
Sources: requirements/cuda.txt
rocm.txt)requirements/rocm.txt adds the AMD-specific stack:
-r common.txt
grpcio==1.78.0
grpcio-reflection==1.78.0
grpcio-tools==1.78.0
numba == 0.61.2
ray[cgraph]>=2.48.0
peft
conch-triton-kernels==1.2.1
timm>=1.0.17
amd-quark>=0.8.99
Unlike the CUDA file, torch is not pinned here — ROCm builds use PyTorch compiled from source in docker/Dockerfile.rocm_base and installed via local wheels.
Sources: requirements/rocm.txt docker/Dockerfile.rocm101-104
rocm-build.txt)requirements/rocm-build.txt is used inside the ROCm build container, where it provides an explicit --extra-index-url for the ROCm wheel index:
-r common.txt
--extra-index-url https://download.pytorch.org/whl/rocm7.1
torch==2.10.0
torchvision==0.25.0
torchaudio==2.10.0
triton==3.6.0
...
amdsmi==7.0.2
Sources: requirements/rocm-build.txt
requirements/build.txt lists packages required only during the build step (python setup.py bdist_wheel):
cmake>=3.26.1
ninja
packaging>=24.2
setuptools>=77.0.3,<81.0.0
setuptools-scm>=8
torch==2.10.0
wheel
jinja2>=3.1.6
regex
build
protobuf >= 5.29.6, ...
grpcio-tools==1.78.0
The comment at the top of build.txt states that its contents must be mirrored in pyproject.toml under [build-system].requires. Both files define the same package list so that both pip install . and direct setup.py invocations get the same build environment.
grpcio-tools is listed because setup.py compiles gRPC protobuf stubs during the build via compile_grpc_protos() setup.py84-140
Sources: requirements/build.txt pyproject.toml1-14 setup.py84-140
test.inrequirements/test.in is the human-maintained list of test requirements. It is not pinned to exact versions. It includes:
pytest, pytest-asyncio, pytest-forked, pytest-rerunfailures, pytest-shard, pytest-timeout, pytest-covlm-eval[api]>=0.4.11, mteb[bm25s]>=2,<3librosa, soundfile, decord, open_clip_torchpeft>=0.15.0, sentence-transformers>=5.2.0, timm>=1.0.17bitsandbytes==0.49.2schemathesis>=3.39.15, grpcio==1.78.0, grpcio-reflection==1.78.0genai_perf>=0.0.8, tritonclient>=2.51.0torch==2.10.0, torchaudio==2.10.0, torchvision==0.25.0Sources: requirements/test.in
test.txtrequirements/test.txt is generated from test.in by uv pip compile and should not be edited manually. The generation command is recorded in its header:
# This file was autogenerated by uv via the following command:
# uv pip compile requirements/test.in -o requirements/test.txt \
# --index-strategy unsafe-best-match \
# --torch-backend cu129 \
# --python-platform x86_64-manylinux_2_28 \
# --python-version 3.12
The locked file contains every transitive dependency with exact version pins, enabling fully reproducible CI test environments. It is regenerated when test.in changes.
Sources: requirements/test.txt1-3
docker/versions.jsonThe docker/versions.json file is the authoritative machine-readable source of all pinned versions used in Docker builds. It is consumed by docker buildx bake via docker/docker-bake.hcl.
This file is auto-generated — do not edit it manually.
generate_versions_json.pytools/generate_versions_json.py parses docker/Dockerfile using dockerfile-parse and extracts all ARG NAME=default declarations into docker/versions.json.
Key functions:
| Function | Purpose |
|---|---|
parse_dockerfile_args(dockerfile_path) | Reads all ARG name=value defaults from the Dockerfile, resolves ${VAR} interpolations |
generate_bake_native_json(args) | Produces the {"variable": {"NAME": {"default": "value"}}} JSON structure |
main() | Orchestrates parsing and writing; --check mode validates the file is in sync |
Version update workflow:
ARG defaults in docker/Dockerfilepython tools/generate_versions_json.pyDockerfile and docker/versions.jsonThis is explicitly described in the Dockerfile header comment docker/Dockerfile9-23
Version tracking flow diagram:
versions.json| Variable | Current Value | Purpose |
|---|---|---|
CUDA_VERSION | 12.9.1 | Base CUDA image version |
PYTHON_VERSION | 3.12 | Python interpreter version |
FLASHINFER_VERSION | 0.6.4 | FlashInfer wheel version |
DEEPGEMM_GIT_REF | 477618cd... | DeepGEMM commit hash |
DEEPEP_COMMIT_HASH | 73b6ea4 | DeepEP commit hash |
BITSANDBYTES_VERSION_X86 | 0.46.1 | BitsAndBytes for x86_64 |
BITSANDBYTES_VERSION_ARM64 | 0.42.0 | BitsAndBytes for ARM64 |
TORCH_CUDA_ARCH_LIST | 7.0 7.5 8.0 8.9 9.0 10.0 12.0 | CUDA architectures compiled for |
Sources: docker/versions.json tools/generate_versions_json.py docker/Dockerfile9-23
use_existing_torch.pyWhen building against nightly PyTorch (PYTORCH_NIGHTLY=1), the version-pinned lines in requirements files conflict with the nightly wheel. use_existing_torch.py solves this by stripping PyTorch-related lines from all requirements/*.txt, requirements/*.in, and pyproject.toml before running uv pip install.
The script targets lines matching these prefixes from TORCH_LIB_PREFIXES use_existing_torch.py9-18:
It accepts a --prefix flag: with --prefix, only lines starting with TORCH_LIB_PREFIXES are removed (safe for requirements files). Without --prefix, any line containing the word torch is removed (used for pyproject.toml).
The Docker build stages invoke it as follows docker/Dockerfile160-172:
if [ "${PYTORCH_NIGHTLY}" = "1" ]; then
python3 use_existing_torch.py --prefix
uv pip install ... -r requirements/cuda.txt \
--extra-index-url .../nightly/cuXXX
else
uv pip install ... -r requirements/cuda.txt \
--extra-index-url .../cuXXX
fi
After stripping, the Dockerfile captures the exact installed nightly versions into torch_lib_versions.txt via uv pip freeze, then propagates them across build stages to ensure version consistency.
Sources: use_existing_torch.py docker/Dockerfile156-180
uvvLLM uses uv (from Astral) instead of pip for faster dependency resolution and installation throughout both Docker builds and local development. Key environment variables set in Docker:
| Variable | Value | Effect |
|---|---|---|
UV_HTTP_TIMEOUT | 500 | Prevents timeout when fetching large wheels |
UV_INDEX_STRATEGY | unsafe-best-match | Allows matching across multiple indexes |
UV_LINK_MODE | copy | Avoids hardlink failures with Docker cache mounts |
The uv binary is installed in the base stage and all subsequent stages inherit it via PATH="/opt/venv/bin:/root/.local/bin:$PATH" docker/Dockerfile124-131
pyproject.toml also declares a uv-specific setting pyproject.toml312-313:
This disables build isolation for torch when using uv, which is necessary because PyTorch does not support PEP 517 build isolation.
Sources: docker/Dockerfile114-131 docker/Dockerfile205-212 pyproject.toml312-313
The following diagram maps the requirements files to the Docker build stages that consume them.
Docker stage to requirements file mapping:
Sources: docker/Dockerfile156-172 docker/Dockerfile202-225 docker/Dockerfile376-389 docker/Dockerfile460-481 docker/Dockerfile720-744 docker/Dockerfile.rocm101-104
| Artifact | Source of Truth | Updated By |
|---|---|---|
| Python runtime deps | requirements/common.txt, requirements/cuda.txt | Manual edits |
| Locked test deps | requirements/test.txt | uv pip compile requirements/test.in |
| Build system deps | requirements/build.txt + pyproject.toml | Manual edits (keep in sync) |
| Docker image versions | docker/Dockerfile ARG defaults | Manual edits to Dockerfile |
versions.json | docker/versions.json | python tools/generate_versions_json.py |
| PyTorch for nightly | Active venv snapshot via uv pip freeze | use_existing_torch.py + Docker build |
Refresh this wiki
This wiki was recently refreshed. Please wait 6 days to refresh again.