This document covers the tool orchestration system and extension points in g4f that enhance chat completions with additional capabilities. The tool system provides:
**Sources:** <FileRef file-url="https://github.com/xtekky/gpt4free/blob/d6dcb36c/g4f/providers/response.py" undefined file-path="g4f/providers/response.py">Hii</FileRef> (referenced), <FileRef file-url="https://github.com/xtekky/gpt4free/blob/d6dcb36c/g4f/tools/run_tools.py#L133-L189" min=133 max=189 file-path="g4f/tools/run_tools.py">Hii</FileRef>
</old_str>
<new_str>
## Thinking & Reasoning Extraction
The `ThinkingProcessor` class (<FileRef file-url="https://github.com/xtekky/gpt4free/blob/d6dcb36c/g4f/tools/run_tools.py#L148-L208" min=148 max=208 file-path="g4f/tools/run_tools.py">Hii</FileRef>) extracts chain-of-thought reasoning from model outputs by detecting `"
"ThinkEnd" --> "NotThinking": "start_time = 0"
"ThinkEnd" --> [*]: "Emit Reasoning(status)"
"NotThinking" --> [*]: "Emit string chunk"
"Thinking" --> "Thinking": "Emit Reasoning(token)"
Sources: g4f/tools/run_tools.py148-208
The process_thinking_chunk() static method (g4f/tools/run_tools.py151-208) handles three cases:
1. Thinking Start (g4f/tools/run_tools.py163-181):
start_time2. Thinking End (g4f/tools/run_tools.py184-202):
</think> tagReasoning(token)start_time > 0Reasoning(status="Thought for {duration}s", is_thinking="</think>")3. Ongoing Thinking (g4f/tools/run_tools.py204-208):
start_time > 0, emit chunk as Reasoning(token)Sources: g4f/tools/run_tools.py151-208
The Reasoning class (g4f/providers/response.py261-295) has these attributes:
| Attribute | Type | Purpose |
|---|---|---|
token | str | Reasoning content token |
label | str | Step label (e.g., "Step 1") |
status | str | Status message (e.g., "Thought for 3.21s") |
is_thinking | str | "" for state transitions |
Sources: g4f/providers/response.py261-295
The tool orchestration system is implemented as a middleware layer that sits between client requests and provider execution. The run_tools module intercepts requests, processes tool calls, enriches messages with additional context, and tracks usage.
Sources: g4f/tools/run_tools.py1-415
The orchestration layer provides two main entry points:
async_iter_run_tools() for asynchronous streaming requestsiter_run_tools() for synchronous streaming requestsBoth functions follow the same processing pipeline but differ in their I/O patterns.
The ToolHandler class (g4f/tools/run_tools.py40-131) processes tool calls embedded in request messages. It validates tool arguments, executes tool-specific logic, and returns modified messages with injected context.
| Tool Name | Function | Purpose | Output |
|---|---|---|---|
search_tool | process_search_tool() | Inject web search results into message content | Modified message + Sources object |
bucket_tool | process_bucket_tool() | Replace bucket references with file content | Modified message with expanded content |
continue_tool | process_continue_tool() | Continue from last message or set provider action | Additional message or modified kwargs |
Tool Call Structure:
Sources: g4f/tools/run_tools.py40-131
Tool names are defined in the TOOL_NAMES constant (g4f/tools/run_tools.py34-38):
These canonical names map internal tool identifiers to their function names in tool call payloads.
Sources: g4f/tools/run_tools.py34-38
Sources: g4f/tools/run_tools.py209-279
web_search kwarg is present, perform search and prepend results to the last messageAuthManager.usage/{date}.jsonlSources: g4f/tools/run_tools.py209-279
The web search tool integrates DuckDuckGo search results into conversation context. It can be triggered explicitly via tool calls or implicitly via the web_search parameter.
| Pattern | Example | Behavior |
|---|---|---|
web_search=True | web_search=True | Extract query from last message |
web_search="query" | web_search="python asyncio" | Use explicit search query |
| Tool call | {"function": {"name": "search_tool"}} | Extract query from message |
| Tool call with args | {"function": {"name": "search_tool", "arguments": {"query": "..."}}} | Use explicit query |
Sources: g4f/tools/run_tools.py192-206 g4f/tools/run_tools.py218-222
The do_search() function (g4f/tools/web_search.py21-54) performs the search and formats results:
CachedSearch.create_async_generator() to retrieve resultsSources object with citation metadataDefault Instructions:
Using the provided web search results, to write a comprehensive reply to the user request.
Make sure to add the sources of cites using [[Number]](Url) notation after the reference.
Example: [[0]](http://google.com)
Sources: g4f/tools/web_search.py16-19 g4f/tools/web_search.py21-54
The bucket tool enables injection of file content into conversations. Files are organized in "buckets" (directories) and referenced by bucket IDs. This system supports document-based conversations without sending large files repeatedly.
~/.config/g4f/buckets/
├── {bucket_id}/
│ ├── files.txt # List of uploaded files
│ ├── downloads.json # URL download specifications
│ ├── plain.cache # Cached plain text content
│ ├── plain_0001.cache # Split cache files
│ ├── spacy_0001.cache # NLP-processed chunks
│ └── {original_files} # Uploaded documents
Sources: g4f/tools/files.py84-88
Bucket content is referenced in messages using JSON syntax:
{"bucket_id": "abc123"}
The tool processor (g4f/tools/run_tools.py82-102) uses regex to find these references and replace them with actual file content:
Sources: g4f/tools/run_tools.py82-102
Sources: g4f/tools/files.py149-213 g4f/tools/files.py215-227 g4f/tools/files.py246-260
The system supports multiple document formats through various libraries:
| Format | Library | Function | Required Package |
|---|---|---|---|
| PyPDF2 | Extract text pages | pypdf2 | |
| pdfplumber | Extract text with layout | pdfplumber | |
| pdfminer | High-level text extraction | pdfminer.six | |
| DOCX | python-docx | Read paragraphs | python-docx |
| DOCX | docx2txt | Simple text extraction | docx2txt |
| ODT | odfpy | Open Document Format | odfpy |
| EPUB | ebooklib | E-book content | ebooklib |
| XLSX | pandas | Spreadsheet rows | openpyxl |
| HTML | BeautifulSoup | Scrape text | beautifulsoup4 |
| ZIP | zipfile | Recursive extraction | (built-in) |
| Plain text | - | Direct read | (built-in) |
Sources: g4f/tools/files.py17-74 g4f/tools/files.py89-122
For URL-based content, the system uses MarkItDown (g4f/tools/files.py427-442) to convert web pages to markdown:
ClientSessionmd.convert_url().md file with source URL annotationSources: g4f/tools/files.py413-488
The continue tool enables automatic continuation of truncated responses. It appends a continuation prompt to the message history or sets a provider-specific action.
1. Message Appending (g4f/tools/run_tools.py68-76):
For most providers, a new user message is added:
2. Provider-Native Continue (g4f/tools/run_tools.py77-78):
For providers like OpenaiAccount and HuggingFaceAPI, set an action:
Sources: g4f/tools/run_tools.py68-79
The ThinkingProcessor class (g4f/tools/run_tools.py133-189) extracts and processes chain-of-thought reasoning from model outputs. It detects `
ThinkingEnd --> NoThinking: calculate duration
ThinkingEnd --> [*]: emit Reasoning chunks
NoThinking --> [*]: emit string chunk
### Reasoning Object Structure
The `Reasoning` response type has the following attributes:
```python
class Reasoning:
token: str = None # Reasoning content
status: str = None # Status message (e.g., "Thought for 3.21s")
label: str = None # Label for reasoning step
is_thinking: str = None # ""
Sources: g4f/providers/response.py (referenced), g4f/tools/run_tools.py133-189
| Input Chunk | Processor State | Output |
|---|---|---|
"Regular text" | start_time=0 | ["Regular text"] |
"BeforeContinue" | start_time>0 | [Reasoning("Final"), Reasoning(status="Thought for 3.21s", is_thinking="</think>"), "Continue"] |
Sources: g4f/tools/run_tools.py137-189
The media processing system handles image, audio, and video content throughout the request/response lifecycle.
The is_data_an_media() function (g4f/image/__init__.py106-118) detects media types from various inputs:
Sources: g4f/image/__init__.py106-118
The EXTENSIONS_MAP constant (g4f/image/__init__.py24-42) defines supported media types:
Sources: g4f/image/__init__.py24-42
Sources: g4f/image/copy_images.py62-108
The save_response_media() function (g4f/image/copy_images.py62-108) handles media saving:
Input Types:
data, mimeType, and transcript keysheaders propertyProcessing Steps:
ImageResponse, AudioResponse, or VideoResponseSources: g4f/image/copy_images.py62-108
The usage tracking system logs token consumption to .usage/{date}.jsonl for analytics and monitoring. Usage is logged asynchronously and synchronously via write_usage() function.
Completion Tokens:
String chunks are counted using byte length approximation (g4f/tools/run_tools.py308 g4f/tools/run_tools.py361):
Prompt Tokens:
The calculate_prompt_tokens() function (g4f/tools/run_tools.py393-407) estimates prompt tokens:
render_messages()Sources: g4f/tools/run_tools.py308 g4f/tools/run_tools.py361 g4f/tools/run_tools.py393-407
Usage is written to JSONL files (g4f/tools/run_tools.py410-415) with one JSON object per line:
Storage Location (g4f/tools/run_tools.py410-411):
Usage is written with aiofile.async_open() for async contexts or standard open() for sync contexts.
Sources: g4f/tools/run_tools.py410-415
The backend API (g4f/gui/server/backend_api.py237-245) provides a /backend-api/v2/usage POST endpoint to log client-side usage:
Sources: g4f/gui/server/backend_api.py237-245
The g4f package provides multiple CLI entry points defined in setup.py119-126 for conversational interfaces and provider-specific tools.
The following console scripts are registered (setup.py119-126):
| Command | Module | Purpose |
|---|---|---|
g4f | g4f.cli:main | Main CLI client for chat and API server |
g4f-mcp | g4f.mcp.server:main | MCP server for AI assistants |
g4f-antigravity | g4f.Provider.needs_auth.Antigravity:cli_main | Antigravity provider CLI |
g4f-geminicli | g4f.Provider.needs_auth.GeminiCLI:cli_main | Gemini provider CLI |
g4f-qwencode | g4f.Provider.qwen.QwenCode:cli_main | Qwen provider CLI |
g4f-github-copilot | g4f.Provider.github.GithubCopilot:cli_main | GitHub Copilot CLI |
Sources: setup.py119-126
The main g4f command (g4f/cli/__init__.py - referenced) supports:
Chat Mode:
API Server Mode:
Sources: README.md196-206
The --edit flag (g4f/cli/client.py - referenced in README) enables file editing:
Example:
Sources: README.md257-268 (referenced from old content)
The CLI detects and processes various input types:
| Input | Detection | Processing |
|---|---|---|
| HTTP(S) URL | startswith("http") | Fetch, convert HTML via MarkItDown if text, add images to media list |
| Local image | is_allowed_extension() | Add to media list |
| Local file | File extension | Read as text, wrap in code fence |
Sources: Referenced from old content (file analysis would require g4f/cli/client.py)
The Model Context Protocol (MCP) server (README.md207-251) exposes g4f tools to AI assistants like Claude Desktop. The server is implemented in g4f/mcp/server.py (referenced) and registered as g4f-mcp CLI command.
Stdio Mode (Default):
HTTP Mode:
Sources: README.md210-230
Add to claude_desktop_config.json (README.md233-244):
Sources: README.md233-244
The MCP server exposes three tools (README.md246-250):
| Tool | Description | Implementation |
|---|---|---|
web_search | Search web using DuckDuckGo | Uses do_search() from g4f/tools/web_search.py |
web_scrape | Extract text from web pages | Uses scraping functions from g4f/tools/files.py |
image_generation | Generate images from prompts | Uses provider image generation |
Sources: README.md246-250
The Client and AsyncClient classes (g4f/client/__init__.py255-270) pass tool parameters to iter_run_tools():
The web_search parameter flows through Completions.create() (g4f/client/__init__.py277-346) to iter_run_tools() (g4f/client/__init__.py308-321).
Sources: g4f/client/__init__.py277-321
The Flask backend (g4f/gui/server/backend_api.py167-231) integrates tools via _create_response_stream():
Tool-Related Parameters:
web_search: Boolean or string extracted from request JSON (g4f/gui/server/backend_api.py294-297)tool_calls: List of tool call objects from requestconversation: JsonConversation for bucket tool stateThese parameters are passed to iter_run_tools() (g4f/gui/server/api.py206).
Sources: g4f/gui/server/backend_api.py167-231 g4f/gui/server/api.py178-311
Some providers have native continue support:
| Provider | Native Support | Implementation |
|---|---|---|
OpenaiAccount | ✓ | Sets action="continue" kwarg (g4f/tools/run_tools.py81-84) |
HuggingFaceAPI | ✓ | Sets action="continue" kwarg (g4f/tools/run_tools.py81-84) |
| Others | Message appending | Adds continuation user message (g4f/tools/run_tools.py76-80) |
Sources: g4f/tools/run_tools.py71-85
Web search failures are logged but don't block the request (g4f/tools/run_tools.py204 g4f/tools/run_tools.py300):
Sources: g4f/tools/run_tools.py200-205 g4f/tools/run_tools.py294-300
Missing dependencies raise MissingRequirementsError (g4f/tools/files.py97 g4f/tools/files.py112):
Sources: g4f/tools/files.py89-122
The tool system maintains a live counter for provider health (g4f/tools/run_tools.py271 g4f/tools/run_tools.py273 g4f/tools/run_tools.py385 g4f/tools/run_tools.py387):
This counter helps the retry system prioritize working providers.
Sources: g4f/tools/run_tools.py270-274 g4f/tools/run_tools.py384-388
Refresh this wiki