Command Line Interface (mineru)

Relevant source files

• docs/en/faq/index.md
• docs/en/reference/output_files.md
• docs/en/usage/cli_tools.md
• docs/en/usage/quick_usage.md
• docs/zh/faq/index.md
• docs/zh/reference/output_files.md
• docs/zh/usage/cli_tools.md
• docs/zh/usage/quick_usage.md
• mineru/backend/hybrid/hybrid_analyze.py
• mineru/backend/hybrid/hybrid_model_output_to_middle_json.py
• mineru/cli/client.py
• mineru/cli/common.py
• mineru/cli/fast_api.py
• mineru/cli/gradio_app.py
• mineru/resources/header.html
• mineru/utils/cli_parser.py
• mineru/utils/pdf_text_tool.py
• mineru/utils/span_pre_proc.py

Purpose and Scope

The mineru command is the primary command-line interface for parsing PDF and image files into structured Markdown and JSON formats. This page covers the main CLI tool implementation, parameter options, backend routing, and execution flow.

For information about other interfaces:

• FastAPI service: see 4.2
• Gradio WebUI: see 4.3
• OpenAI-compatible server: see 4.4
• Python API integration: see 4.5

CLI Entry Point Architecture

The main CLI entry point is implemented in mineru/cli/client.py21-223 using the Click framework. The command accepts arguments, configures the environment, loads files, and delegates to the core parsing orchestration.

Sources: mineru/cli/client.py21-223 mineru/cli/common.py414-484

Command Parameters

Required Parameters

Parameter	Short	Type	Description
--path	-p	Path	Input PDF/image file or directory (required)
--output	-o	Path	Output directory (required)

Backend Selection

Parameter	Short	Type	Default	Valid Options
--backend	-b	Choice	hybrid-auto-engine	pipeline, vlm-http-client, hybrid-http-client, vlm-auto-engine, hybrid-auto-engine

Document Processing Options

Parameter	Short	Type	Default	Description	Applicable Backends
--method	-m	Choice	auto	Parsing method: auto, txt, ocr	pipeline, hybrid-*
--lang	-l	Choice	ch	OCR language code	pipeline, hybrid-*
--start	-s	int	0	Starting page number (0-indexed)	All
--end	-e	int	None	Ending page number (0-indexed)	All
--formula	-f	bool	True	Enable formula parsing	All
--table	-t	bool	True	Enable table parsing	All
--url	-u	str	None	Server URL for http-client backends	*-http-client

Hardware Configuration

Parameter	Short	Type	Default	Description	Applicable Backends
--device	-d	str	auto-detect	Device mode: cpu, cuda, cuda:0, npu, npu:0, mps	pipeline
--vram		int	auto-detect	GPU VRAM limit per process (GB)	pipeline
--source		Choice	huggingface	Model source: huggingface, modelscope, local	All

Other Options

Parameter	Short	Description
--version	-v	Display version and exit
--help		Show help message and exit

Sources: mineru/cli/client.py23-151 docs/en/usage/cli_tools.md7-28

Execution Flow Diagram

Sources: mineru/cli/client.py154-221 mineru/cli/common.py414-556

Backend Routing Logic

The CLI routes requests to different processing backends based on the --backend parameter. The routing logic includes prefix stripping and engine resolution.

Sources: mineru/cli/common.py414-556

Backend Prefix Handling

The implementation strips backend prefixes to normalize engine names:

• Line mineru/cli/common.py447-448: Strips vlm- prefix
• Line mineru/cli/common.py465-466: Strips hybrid- prefix

After stripping, if the backend is auto-engine, the system calls get_vlm_engine() with inference_engine='auto' to automatically select the best available engine based on hardware and installed dependencies.

Sources: mineru/cli/common.py447-474 mineru/cli/common.py520-545

File Input Processing

Supported File Types

The CLI processes PDF and image files. Supported formats are defined in mineru/cli/common.py27-28:

File Loading Flow

Sources: mineru/cli/client.py213-220 mineru/cli/common.py32-43

PDF Page Range Extraction

When --start and/or --end parameters are provided, the CLI extracts a subset of pages using pypdfium2:

The function _prepare_pdf_bytes() at mineru/cli/common.py85-91 calls convert_pdf_bytes_to_bytes_by_pypdfium2() which:

1. Opens the PDF with pdfium.PdfDocument(pdf_bytes)
2. Creates a new PDF document
3. Imports pages in the specified range with error handling
4. Returns the new PDF as bytes

Page import failures are logged but don't stop processing:

Sources: mineru/cli/common.py54-92

Environment Variable Configuration

Hardware Detection and Setup

For non-http-client backends, the CLI automatically configures hardware-related environment variables:

Sources: mineru/cli/client.py163-182

Backend-Specific Environment Variables

Different backends require specific environment variable configurations set in do_parse():

VLM Backend (vlm-*):

Hybrid Backend (hybrid-*):

Sources: mineru/cli/common.py456-457 mineru/cli/common.py475-476 mineru/cli/common.py529-530 mineru/cli/common.py547-548

Global Environment Settings

The CLI also sets some global environment variables:

• TOKENIZERS_PARALLELISM = "false" - Disables tokenizer parallelism to avoid multiprocessing issues mineru/cli/common.py30
• Special handling for MACA devices: Disables cudnn when MINERU_LMDEPLOY_DEVICE=maca mineru/cli/common.py22-24

Sources: mineru/cli/common.py22-30

Output Directory Structure

Directory Creation

The prepare_env() function creates the output directory structure:

For hybrid backend, the parse_method is modified to hybrid_{parse_method}.

Sources: mineru/cli/common.py46-51

Output File Structure

output_dir/
└── {pdf_file_name}/
    └── {parse_method}/           # e.g., "auto", "ocr", "vlm", "hybrid_auto"
        ├── {pdf_file_name}.md
        ├── {pdf_file_name}_content_list.json
        ├── {pdf_file_name}_content_list_v2.json  (VLM/Hybrid only)
        ├── {pdf_file_name}_middle.json
        ├── {pdf_file_name}_model.json
        ├── {pdf_file_name}_layout.pdf
        ├── {pdf_file_name}_span.pdf              (Pipeline only)
        ├── {pdf_file_name}_origin.pdf
        └── images/
            └── *.jpg

Output Generation Process

The _process_output() function at mineru/cli/common.py94-168 handles all output file generation based on flags:

Sources: mineru/cli/common.py94-168

Default Output Flags

By default in do_parse() and aio_do_parse():

• f_draw_layout_bbox = True
• f_draw_span_bbox = True
• f_dump_md = True
• f_dump_middle_json = True
• f_dump_model_output = True
• f_dump_orig_pdf = True
• f_dump_content_list = True
• f_make_md_mode = MakeMode.MM_MD

Sources: mineru/cli/common.py424-431

Advanced Features

Extra Argument Parsing

The CLI supports passing additional arguments not defined as Click options through the context:

The arg_parse() function in mineru/utils/cli_parser.py4-38 parses --param value style arguments from ctx.args, converting values to appropriate types (bool, int, float, str). This mechanism allows passing vLLM/LMDeploy-specific parameters directly through the CLI.

Sources: mineru/cli/client.py161 mineru/utils/cli_parser.py4-38

Async vs Sync Mode

The CLI provides both synchronous and asynchronous entry points:

• do_parse() - Synchronous, returns when complete mineru/cli/common.py414-484
• aio_do_parse() - Asynchronous, can be awaited mineru/cli/common.py486-556

Both functions have identical signatures and similar logic but differ in:

• How they call backend functions (sync vs async)
• Validation of inference engine compatibility (vllm-engine vs vllm-async-engine)

The CLI itself uses do_parse() synchronously, while other interfaces like Gradio and FastAPI may use aio_do_parse().

Sources: mineru/cli/common.py414-556

Backend Mode Validation

The implementation enforces strict mode compatibility:

In do_parse (sync mode):

• Raises exception if vlm-vllm-async-engine is used mineru/cli/common.py450-451
• Raises exception if hybrid-vllm-async-engine is used mineru/cli/common.py468-470

In aio_do_parse (async mode):

• Raises exception if vlm-vllm-engine is used mineru/cli/common.py523-524
• Raises exception if hybrid-vllm-engine is used mineru/cli/common.py541-542

Sources: mineru/cli/common.py450-451 mineru/cli/common.py468-470 mineru/cli/common.py523-524 mineru/cli/common.py541-542

Logging Configuration

All CLI tools follow a consistent logging setup pattern using loguru:

This allows users to control verbosity through the MINERU_LOG_LEVEL environment variable (e.g., DEBUG, INFO, WARNING, ERROR).

Sources: mineru/cli/client.py9-11 mineru/cli/gradio_app.py16-18 mineru/cli/fast_api.py19-21

Common Usage Patterns

Basic Single File Parsing

Directory Batch Processing

Backend Selection Examples

Page Range and Options

Hardware Configuration

Sources: docs/en/usage/cli_tools.md1-61 docs/zh/usage/cli_tools.md1-56 docs/en/usage/quick_usage.md10-54

Integration with System Architecture

The CLI serves as the primary user-facing entry point that orchestrates the complete parsing pipeline:

The CLI implementation in client.py is a thin wrapper that:

1. Parses command-line arguments
2. Configures environment variables
3. Loads and validates input files
4. Delegates to do_parse() in common.py

The common.py module contains the core orchestration logic shared by all interfaces (CLI, API, Gradio), ensuring consistent behavior across different entry points.

Sources: mineru/cli/client.py1-224 mineru/cli/common.py1-570