Test Framework Infrastructure

Relevant source files

• Release.md
• client/connector.go
• go.mod
• go.sum
• hack/run-e2e.sh
• pkg/util/version/version.go
• pkg/util/xlog/log_writer.go

Purpose and Scope

This document describes the testing framework infrastructure used in the frp codebase. It covers the Ginkgo/Gomega testing framework, test execution utilities, mock server implementations, and the supporting infrastructure for running end-to-end tests. For information about the actual test scenarios and test coverage, see End-to-End Testing. For code quality enforcement tools, see Code Quality and Linting.

Framework Overview

The frp project uses Ginkgo v2 as its primary BDD-style testing framework, paired with Gomega for assertions. This combination provides expressive, readable test specifications and comprehensive matcher support for validating complex proxy behaviors.

Key Components:

• Ginkgo v2.23.4: BDD-style test framework with parallel execution support
• Gomega v1.36.3: Assertion and matcher library
• Testify v1.10.0: Additional assertion utilities
• Custom test utilities: Helper functions for frpc/frps lifecycle management
• Mock servers: Test doubles for external services

Sources: go.mod13-14 go.mod25

Framework Architecture

Sources: hack/run-e2e.sh1-35 go.mod13-14 go.mod25

Ginkgo Test Framework

Ginkgo is a BDD-style Go testing framework that provides rich test organization capabilities, parallel execution, and expressive test specifications.

Core Components

Test Suite Organization:

• Describe blocks: Group related test cases
• Context blocks: Describe specific scenarios
• It blocks: Individual test specifications
• BeforeEach/AfterEach: Setup and teardown hooks
• BeforeSuite/AfterSuite: Suite-level initialization

Parallel Execution: The framework supports running tests in parallel with configurable concurrency:

ginkgo -nodes=${concurrency} --poll-progress-after=60s

The default concurrency is 16 nodes, which can be overridden via the CONCURRENCY environment variable.

Progress Reporting: The --poll-progress-after=60s flag enables progress reporting for long-running tests, printing status updates every 60 seconds to detect hanging tests.

Sources: hack/run-e2e.sh29-34

Test Execution Flow

Sources: hack/run-e2e.sh34

Gomega Assertion Library

Gomega provides expressive matchers for test assertions, enabling readable and maintainable test code.

Matcher Categories

Basic Matchers:

• Equal(expected): Value equality
• BeNil(): Nil check
• BeTrue()/BeFalse(): Boolean assertions
• HaveLen(length): Collection length
• ContainSubstring(substring): String matching

Network Matchers:

• Custom matchers for connection validation
• HTTP response matchers
• Port availability checks

Async Matchers:

• Eventually(func): Retry until success or timeout
• Consistently(func): Verify condition remains true
• Configurable timeouts and poll intervals

Usage Patterns

Tests typically use Eventually for operations that may take time:

Sources: go.mod14

Test Execution Infrastructure

Test Runner Script

The hack/run-e2e.sh1-35 script provides a convenient wrapper around the Ginkgo CLI with environment-based configuration.

Environment Variables:

Variable	Default	Purpose
DEBUG	false	Enable debug mode for verbose output
LOG_LEVEL	debug	Set log level (trace, debug, info, warn, error)
FRPC_PATH	${ROOT}/bin/frpc	Path to frpc binary
FRPS_PATH	${ROOT}/bin/frps	Path to frps binary
CONCURRENCY	16	Number of parallel test nodes

Script Workflow:

1. Ginkgo Installation Check: Verifies ginkgo CLI is available, installs if missing
2. Environment Configuration: Processes environment variables and sets defaults
3. Binary Path Resolution: Determines frpc/frps binary locations
4. Test Execution: Invokes Ginkgo with appropriate flags

Sources: hack/run-e2e.sh6-34

Process Manager Utilities

Test utilities manage frpc/frps process lifecycles:

Responsibilities:

• Launch server/client processes with generated configurations
• Manage process cleanup on test completion
• Capture and redirect process output to test logs
• Detect process crashes and report failures
• Support graceful shutdown and forced termination

Port Allocation

To support parallel test execution, the framework includes dynamic port allocation:

Features:

• Thread-safe port assignment
• Port range reservation for each test node
• Automatic cleanup on test completion
• Conflict detection and retry logic

Test Configuration Management

Dynamic Configuration Generation

Test utilities generate frpc/frps configuration files dynamically:

Configuration Templates:

• Base server configuration with dynamic port binding
• Per-test proxy configurations
• Authentication and security settings
• Transport protocol selection

Format Support: The configuration generator supports multiple formats (TOML, YAML, JSON, INI) to test configuration loading across all supported formats.

Log Integration

The test framework integrates with frp's logging system via pkg/util/xlog/log_writer.go1-66:

LogWriter Types:

• NewTraceWriter(xl): Trace-level output for detailed debugging
• NewDebugWriter(xl): Debug-level for test diagnostics
• NewInfoWriter(xl): Information-level for test progress
• NewWarnWriter(xl): Warning-level for non-critical issues
• NewErrorWriter(xl): Error-level for failures

The yamux multiplexer uses NewTraceWriter to minimize log noise during normal test execution while retaining full diagnostics when needed:

Sources: pkg/util/xlog/log_writer.go19-66 client/connector.go118

Mock Server Infrastructure

Mock HTTP Server

Mock HTTP servers use gorilla/mux for routing:

Capabilities:

• Custom request handlers for testing specific behaviors
• Header validation
• Body content verification
• Response code simulation
• Request counting for assertion

Example Use Cases:

• Testing HTTP proxy functionality
• Validating HTTP header manipulation
• Testing HTTP authentication flows
• Simulating backend service responses

Sources: go.mod10

Mock TCP/UDP Servers

TCP Mock Server:

• Accept connections and echo data
• Implement custom protocols for testing
• Validate connection sequence
• Test connection pooling behavior

UDP Mock Server:

• Receive and respond to UDP packets
• Validate packet headers (e.g., UDP proxy protocol)
• Test packet loss scenarios
• Verify session management

WebSocket Mock Server

Uses gorilla/websocket for testing WebSocket transport:

Test Scenarios:

• WebSocket upgrade handshake
• TLS-enabled WebSocket (WSS)
• Custom TLS header byte handling
• Connection persistence

Sources: go.mod11

Test Utilities and Helpers

Connection Testing Utilities

Process Lifecycle Management

Process Control Interface:

• Start(): Launch frpc/frps with configuration
• Stop(): Graceful shutdown
• Kill(): Force termination
• Wait(): Wait for process exit
• IsRunning(): Process status check

Output Capture:

• Redirect stdout/stderr to test logger
• Preserve output for failure analysis
• Support both streaming and buffered capture

Version Compatibility Testing

The framework uses the version information from pkg/util/version/version.go15-21 to support backward compatibility testing:

Version Testing:

• Test current version against previous releases
• Validate protocol compatibility
• Ensure migration paths work correctly

Sources: pkg/util/version/version.go15-21

Parallel Test Execution

Node-Based Parallelism

Ginkgo distributes tests across multiple nodes (processes):

Coordination:

• Each node receives a subset of test specs
• Nodes run independently and report results
• Final aggregation produces unified test report

Resource Management:

• Port ranges assigned per node to avoid conflicts
• Temporary file directories isolated by node
• Process cleanup scoped to node lifetime

Test Isolation

Isolation Strategies:

• Each test spec gets unique ports
• Separate configuration files per test
• Independent frpc/frps instances
• Cleanup hooks ensure no state leakage

Sources: hack/run-e2e.sh29-34

Debugging and Testing Support

Debug Mode

Enable via DEBUG=true environment variable:

Debug Features:

• Verbose process output
• Extended logging from frpc/frps
• Preserved temporary files
• Step-by-step execution logging

Log Levels: Configurable via LOG_LEVEL environment variable:

• trace: Maximum verbosity, including protocol-level details
• debug: Detailed operational information
• info: General progress information
• warn: Warning messages only
• error: Error messages only

Sources: hack/run-e2e.sh12-19

Test Failure Diagnostics

Failure Information Captured:

• Process exit codes
• Full stdout/stderr from frpc/frps
• Configuration files used
• Network state at failure time
• Timing information

Gomega Failure Messages: Gomega provides detailed failure messages including:

• Expected vs actual values
• Stack traces
• Custom failure messages
• Diff output for complex comparisons

Testing Transport Protocols

The test framework validates all transport protocols supported by the Connector abstraction:

Transport Protocol Coverage

TCP Transport:

• Direct TCP connections
• TLS-encrypted connections
• Custom TLS header byte handling

WebSocket Transport:

• Standard WebSocket upgrade
• WSS (WebSocket Secure)
• Connection persistence

QUIC Transport:

• QUIC stream multiplexing
• TLS configuration validation
• Keep-alive behavior

KCP Transport:

• Low-latency UDP-based transport
• Reliability testing
• Performance characteristics

Sources: client/connector.go37-227

TLS Testing Utilities

TLS Configuration Validation:

• Certificate verification
• Server name validation
• Custom CA certificate support
• TLS version enforcement

Custom TLS Features:

• First byte customization testing
• TLS header manipulation
• Protocol negotiation validation

Sources: client/connector.go69-104 client/connector.go147-196

Test Suite Organization

Integration with CI/CD

The test framework integrates with the CI/CD pipeline documented in Development and Deployment:

CI Execution:

• GitHub Actions runs tests on multiple platforms
• CircleCI provides additional test coverage
• Test results reported to PR checks
• Failed tests block merges

Platform Coverage:

• Linux (amd64, arm64)
• macOS
• Windows
• Docker containers

Sources: See CI/CD Pipeline for details

Best Practices

Test Organization

• Group related tests in Describe blocks
• Use Context to describe different scenarios
• Keep It blocks focused on single behaviors
• Use BeforeEach for test setup, not BeforeSuite

Resource Management

• Always clean up processes in AfterEach
• Release ports promptly
• Remove temporary files
• Close connections explicitly

Assertions

• Use Eventually for asynchronous operations
• Set appropriate timeouts (default may be too short)
• Provide descriptive failure messages
• Validate all important aspects of behavior

Debugging

• Enable debug mode for failing tests
• Check process output in logs
• Verify configuration files
• Use step-through execution when needed