Testing Strategy

Relevant source files

This page describes the test infrastructure for the Langflow monorepo: how tests are organized, what tools are used, how they are run locally, and how CI executes them. For information about the CI pipeline itself (job definitions, path filtering, conditional triggers), see CI Pipeline. For code quality tools such as linting and type-checking, see Code Quality Tools.

Overview

The Langflow repository has three distinct test layers:

Layer	Technology	Location	Purpose
Backend unit tests	pytest	`src/backend/tests/unit/`	Isolated logic tests, no external services
Backend integration tests	pytest	`src/backend/tests/integration/`	Full-stack API tests against a live app instance
LFX package tests	pytest	`src/lfx/tests/unit/`	Tests for the `lfx` lightweight executor package
Frontend unit tests	Jest	`src/frontend/src/**/__tests__/`	React component and utility logic tests
Frontend E2E tests	Playwright	`src/frontend/tests/`	Browser-level flow interaction tests

Backend Tests

Framework and Configuration

Backend tests use pytest. All configuration lives in the top-level pyproject.toml.

pyproject.toml164-185

Key settings:

Setting	Value	Effect
`testpaths`	`["src/backend/tests", "src/lfx/tests"]`	Directories scanned for tests
`timeout`	`150` seconds	Per-test timeout via `pytest-timeout`
`timeout_method`	`signal`	Uses UNIX signal (not thread) for timeout
`asyncio_mode`	`auto`	All `async def` test functions run as async automatically
`filterwarnings`	ignore `DeprecationWarning`, `ResourceWarning`	Suppresses noisy warnings from dependencies

Test Markers

Markers allow selective test execution.

pyproject.toml174-182

markers:
  unit              – pure unit tests
  integration       – full-stack integration tests
  api_key_required  – requires external API credentials
  no_blockbuster    – opt-out from blocking I/O detection
  benchmark         – performance benchmarks
  slow              – long-running tests
  async_test        – legacy async marker (asyncio_mode=auto supersedes this)

Tests marked api_key_required are excluded from standard CI runs using -m 'not api_key_required'.

Pytest Plugins Used

pyproject.toml24-75

Plugin	Role
`pytest-asyncio`	Async test execution; `asyncio_mode=auto` is set
`pytest-xdist`	Parallel execution via `-n auto`
`pytest-split`	Time-based test sharding for CI (`--splits N --group G`)
`pytest-cov`	Coverage collection and reporting
`pytest-mock`	Mocking utilities (`mocker` fixture)
`pytest-rerunfailures`	Retry flaky tests (`--reruns 5`)
`pytest-instafail`	Print failures immediately, not at end
`pytest-timeout`	Enforce per-test wall-clock limits
`blockbuster`	Detect synchronous blocking I/O in async code
`hypothesis`	Property-based testing
`respx`	Mock HTTPX async requests
`asgi-lifespan`	Run FastAPI lifespan in test context
`locust`	Load/stress testing (`src/backend/tests/locust/`)

Test Fixtures (`conftest.py`)

src/backend/tests/conftest.py1-60

The main conftest.py provides the following fixtures:

Test Application Fixture Diagram

Key fixtures:

Fixture	Scope	Description
`client`	function	Synchronous `TestClient` wrapping the app
`async_client`	function	`AsyncClient` with `ASGITransport`
`logged_in_headers`	function	Auth headers for a test user
`flow_data`	function	A minimal valid flow dict
`created_api_key`	function	An `ApiKey` row seeded into the test DB

The test database is an isolated SQLite instance per test run. The app is instantiated using create_app() from langflow.main and wrapped in LifespanManager to trigger startup and shutdown hooks.

Test Directory Structure

src/backend/tests/
├── conftest.py               # Shared fixtures
├── .test_durations           # Historical timing data for pytest-split
├── unit/
│   ├── test_database.py
│   ├── test_initial_setup.py
│   ├── test_graph/
│   ├── test_components/
│   ├── template/
│   │   └── test_starter_projects.py
│   └── ...
├── integration/
│   ├── test_flows.py
│   ├── test_chat.py
│   └── ...
└── locust/                   # Load test scripts

src/lfx/tests/
└── unit/
    └── ...                   # LFX package unit tests

Sources: src/backend/tests/conftest.py1-60 pyproject.toml164-185 Makefile144-206

Running Backend Tests Locally

Makefile144-206

Command	What it runs
`make unit_tests`	Unit tests with parallel execution (`-n auto`)
`make integration_tests`	All integration tests
`make integration_tests_no_api_keys`	Integration tests excluding `api_key_required`
`make integration_tests_api_keys`	Only `api_key_required` integration tests
`make lfx_tests`	LFX package tests with coverage
`make template_tests`	Starter project template tests
`make coverage`	Unit tests + HTML coverage report

Example invocation for unit tests with extra args:

make unit_tests args="-x -vv --lf"

Coverage

pyproject.toml187-204

Coverage is collected from src/backend/base/langflow/ and excludes:

Alembic migration scripts (*/alembic/*)
Test files themselves (tests/*)
__init__.py files

Coverage is reported as HTML in the coverage/ directory and uploaded to Codecov from Python 3.10 CI runs.

Frontend Tests

Unit Tests (Jest)

src/frontend/package.json106-126

Jest is configured with ts-jest for TypeScript transpilation and jest-environment-jsdom for a browser-like DOM. Testing utilities include @testing-library/react, @testing-library/user-event, and @testing-library/jest-dom.

Script	Command
`npm test`	`jest`
`npm run test:coverage`	`jest --coverage`
`npm run test:watch`	`jest --watch`

Relevant dev dependencies:

Package	Role
`jest`	Test runner
`ts-jest`	TypeScript support in Jest
`jest-environment-jsdom`	Simulated browser DOM
`@testing-library/react`	React component rendering utilities
`@testing-library/user-event`	Simulated user interactions
`@testing-library/jest-dom`	Custom matchers (`.toBeInTheDocument()`, etc.)
`jest-junit`	JUnit XML reports for CI

Sources: src/frontend/package.json107-177

End-to-End Tests (Playwright)

.github/workflows/typescript_test.yml1-80

E2E tests use @playwright/test and are located in src/frontend/tests/. Tests are organized into named suites, each tagged with a @<suitename> annotation in test titles.

Test Suites and Tags

Suite Name	Playwright Tag	Triggered When
`release`	`@release`	Release builds; always includes all tagged tests
`components`	`@components`	Component files changed
`starter-projects`	`@starter-projects`	Starter project files changed
`workspace`	`@workspace`	Workspace files changed
`api`	`@api`	API-related files changed
`database`	`@database`	Database files changed
`mainpage`	`@mainpage`	Main page files changed
`development`	`@development`	Development-related files changed

Suite selection is driven by path-based change detection using dorny/paths-filter and the filter definitions in .github/changes-filter.yaml.

.github/changes-filter.yaml1-50

Test Sharding

.github/workflows/typescript_test.yml230-320

Playwright tests are sharded across multiple runner instances using a matrix strategy. Each shard runs a subset of tests in parallel. The determine-test-suite job computes which suites and how many shards are needed, then passes a matrix to the run-playwright job.

Playwright CI Execution Flow

Sources: .github/workflows/typescript_test.yml74-320

CI Test Execution

Both backend and frontend test workflows are invoked as reusable sub-workflows by ci.yml.

.github/workflows/ci.yml1-70 .github/workflows/python_test.yml1-110

Backend CI Matrix

.github/workflows/python_test.yml55-100

The python_test.yml workflow runs unit tests across a matrix of Python versions and test split groups:

matrix:
  python-version: [3.10, 3.11, 3.12, 3.13]
  splitCount: [5]
  group: [1, 2, 3, 4, 5]

This produces 20 parallel jobs (4 Python versions × 5 groups). Historical test durations stored in src/backend/tests/.test_durations are used by pytest-split to distribute tests evenly.

Each job uses:

make unit_tests args="-x -vv --splits 5 --group N --reruns 5 --cov ..."

The nick-fields/retry action wraps the run with 2 attempts and a 12-minute timeout per attempt.

Full Test Execution Flow

CI Test Workflow Diagram

Sources: .github/workflows/ci.yml170-350 .github/workflows/python_test.yml55-110 .github/workflows/typescript_test.yml74-350

Test Configuration Summary

`pyproject.toml` pytest settings (root)

pyproject.toml164-185

`src/backend/base/pyproject.toml` pytest settings

src/backend/base/pyproject.toml144-153

This configuration is used when running tests directly from within the langflow-base package directory (less common in CI, which uses the root configuration).

Blocking I/O Detection

The blockbuster library is used to detect synchronous blocking calls (e.g., time.sleep, synchronous file reads) made from within async code. It is activated by the blockbuster_ctx context manager in conftest.py.

src/backend/tests/conftest.py15-25

Tests that legitimately need to call blocking code can opt out by applying the @pytest.mark.no_blockbuster marker.

Load Testing

A locust-based load test suite lives under src/backend/tests/locust/. It is not part of the standard CI test run and must be invoked manually using make load_test_* targets defined in the Makefile.

Makefile455-530

Load test targets include:

load_test_setup – starts a local Langflow server for load testing
load_test_run – runs locust against the running server
load_test_clean – tears down the load test environment

Ruff Overrides for Test Files

Test files are given relaxed linting rules in pyproject.toml to allow common test patterns:

pyproject.toml337-359

Override	Reason
`S101`	Allow `assert` statements
`PLR2004`	Allow magic value comparisons
`SLF001`	Allow private member access
`BLE001`	Allow broad exception catching
`D1`	No docstring requirements

Testing Strategy

Relevant source files

Overview

The Langflow repository has three distinct test layers:

Layer	Technology	Location	Purpose
Backend unit tests	pytest	`src/backend/tests/unit/`	Isolated logic tests, no external services
Backend integration tests	pytest	`src/backend/tests/integration/`	Full-stack API tests against a live app instance
LFX package tests	pytest	`src/lfx/tests/unit/`	Tests for the `lfx` lightweight executor package
Frontend unit tests	Jest	`src/frontend/src/**/__tests__/`	React component and utility logic tests
Frontend E2E tests	Playwright	`src/frontend/tests/`	Browser-level flow interaction tests

Backend Tests

Framework and Configuration

Backend tests use pytest. All configuration lives in the top-level pyproject.toml.

pyproject.toml164-185

Key settings:

Setting	Value	Effect
`testpaths`	`["src/backend/tests", "src/lfx/tests"]`	Directories scanned for tests
`timeout`	`150` seconds	Per-test timeout via `pytest-timeout`
`timeout_method`	`signal`	Uses UNIX signal (not thread) for timeout
`asyncio_mode`	`auto`	All `async def` test functions run as async automatically
`filterwarnings`	ignore `DeprecationWarning`, `ResourceWarning`	Suppresses noisy warnings from dependencies

Test Markers

Markers allow selective test execution.

pyproject.toml174-182

markers:
  unit              – pure unit tests
  integration       – full-stack integration tests
  api_key_required  – requires external API credentials
  no_blockbuster    – opt-out from blocking I/O detection
  benchmark         – performance benchmarks
  slow              – long-running tests
  async_test        – legacy async marker (asyncio_mode=auto supersedes this)

Tests marked api_key_required are excluded from standard CI runs using -m 'not api_key_required'.

Pytest Plugins Used

pyproject.toml24-75

Plugin	Role
`pytest-asyncio`	Async test execution; `asyncio_mode=auto` is set
`pytest-xdist`	Parallel execution via `-n auto`
`pytest-split`	Time-based test sharding for CI (`--splits N --group G`)
`pytest-cov`	Coverage collection and reporting
`pytest-mock`	Mocking utilities (`mocker` fixture)
`pytest-rerunfailures`	Retry flaky tests (`--reruns 5`)
`pytest-instafail`	Print failures immediately, not at end
`pytest-timeout`	Enforce per-test wall-clock limits
`blockbuster`	Detect synchronous blocking I/O in async code
`hypothesis`	Property-based testing
`respx`	Mock HTTPX async requests
`asgi-lifespan`	Run FastAPI lifespan in test context
`locust`	Load/stress testing (`src/backend/tests/locust/`)

Test Fixtures (`conftest.py`)

src/backend/tests/conftest.py1-60

The main conftest.py provides the following fixtures:

Test Application Fixture Diagram

Key fixtures:

Fixture	Scope	Description
`client`	function	Synchronous `TestClient` wrapping the app
`async_client`	function	`AsyncClient` with `ASGITransport`
`logged_in_headers`	function	Auth headers for a test user
`flow_data`	function	A minimal valid flow dict
`created_api_key`	function	An `ApiKey` row seeded into the test DB

Test Directory Structure

src/backend/tests/
├── conftest.py               # Shared fixtures
├── .test_durations           # Historical timing data for pytest-split
├── unit/
│   ├── test_database.py
│   ├── test_initial_setup.py
│   ├── test_graph/
│   ├── test_components/
│   ├── template/
│   │   └── test_starter_projects.py
│   └── ...
├── integration/
│   ├── test_flows.py
│   ├── test_chat.py
│   └── ...
└── locust/                   # Load test scripts

src/lfx/tests/
└── unit/
    └── ...                   # LFX package unit tests

Sources: src/backend/tests/conftest.py1-60 pyproject.toml164-185 Makefile144-206

Running Backend Tests Locally

Makefile144-206

Command	What it runs
`make unit_tests`	Unit tests with parallel execution (`-n auto`)
`make integration_tests`	All integration tests
`make integration_tests_no_api_keys`	Integration tests excluding `api_key_required`
`make integration_tests_api_keys`	Only `api_key_required` integration tests
`make lfx_tests`	LFX package tests with coverage
`make template_tests`	Starter project template tests
`make coverage`	Unit tests + HTML coverage report

Example invocation for unit tests with extra args:

make unit_tests args="-x -vv --lf"

Coverage

pyproject.toml187-204

Coverage is collected from src/backend/base/langflow/ and excludes:

Alembic migration scripts (*/alembic/*)
Test files themselves (tests/*)
__init__.py files

Coverage is reported as HTML in the coverage/ directory and uploaded to Codecov from Python 3.10 CI runs.

Frontend Tests

Unit Tests (Jest)

src/frontend/package.json106-126

Script	Command
`npm test`	`jest`
`npm run test:coverage`	`jest --coverage`
`npm run test:watch`	`jest --watch`

Relevant dev dependencies:

Package	Role
`jest`	Test runner
`ts-jest`	TypeScript support in Jest
`jest-environment-jsdom`	Simulated browser DOM
`@testing-library/react`	React component rendering utilities
`@testing-library/user-event`	Simulated user interactions
`@testing-library/jest-dom`	Custom matchers (`.toBeInTheDocument()`, etc.)
`jest-junit`	JUnit XML reports for CI

Sources: src/frontend/package.json107-177

End-to-End Tests (Playwright)

.github/workflows/typescript_test.yml1-80

E2E tests use @playwright/test and are located in src/frontend/tests/. Tests are organized into named suites, each tagged with a @<suitename> annotation in test titles.

Test Suites and Tags

Suite Name	Playwright Tag	Triggered When
`release`	`@release`	Release builds; always includes all tagged tests
`components`	`@components`	Component files changed
`starter-projects`	`@starter-projects`	Starter project files changed
`workspace`	`@workspace`	Workspace files changed
`api`	`@api`	API-related files changed
`database`	`@database`	Database files changed
`mainpage`	`@mainpage`	Main page files changed
`development`	`@development`	Development-related files changed

Suite selection is driven by path-based change detection using dorny/paths-filter and the filter definitions in .github/changes-filter.yaml.

.github/changes-filter.yaml1-50

Test Sharding

.github/workflows/typescript_test.yml230-320

Playwright CI Execution Flow

Sources: .github/workflows/typescript_test.yml74-320

CI Test Execution

Both backend and frontend test workflows are invoked as reusable sub-workflows by ci.yml.

.github/workflows/ci.yml1-70 .github/workflows/python_test.yml1-110

Backend CI Matrix

.github/workflows/python_test.yml55-100

The python_test.yml workflow runs unit tests across a matrix of Python versions and test split groups:

matrix:
  python-version: [3.10, 3.11, 3.12, 3.13]
  splitCount: [5]
  group: [1, 2, 3, 4, 5]

This produces 20 parallel jobs (4 Python versions × 5 groups). Historical test durations stored in src/backend/tests/.test_durations are used by pytest-split to distribute tests evenly.

Each job uses:

make unit_tests args="-x -vv --splits 5 --group N --reruns 5 --cov ..."

The nick-fields/retry action wraps the run with 2 attempts and a 12-minute timeout per attempt.

Full Test Execution Flow

CI Test Workflow Diagram

Sources: .github/workflows/ci.yml170-350 .github/workflows/python_test.yml55-110 .github/workflows/typescript_test.yml74-350

Test Configuration Summary

`pyproject.toml` pytest settings (root)

pyproject.toml164-185

`src/backend/base/pyproject.toml` pytest settings

src/backend/base/pyproject.toml144-153

This configuration is used when running tests directly from within the langflow-base package directory (less common in CI, which uses the root configuration).

Blocking I/O Detection

src/backend/tests/conftest.py15-25

Tests that legitimately need to call blocking code can opt out by applying the @pytest.mark.no_blockbuster marker.

Load Testing

Makefile455-530

Load test targets include:

load_test_setup – starts a local Langflow server for load testing
load_test_run – runs locust against the running server
load_test_clean – tears down the load test environment

Ruff Overrides for Test Files

Test files are given relaxed linting rules in pyproject.toml to allow common test patterns:

pyproject.toml337-359

Override	Reason
`S101`	Allow `assert` statements
`PLR2004`	Allow magic value comparisons
`SLF001`	Allow private member access
`BLE001`	Allow broad exception catching
`D1`	No docstring requirements

Testing Strategy

Overview

Backend Tests

Framework and Configuration

Test Markers

Pytest Plugins Used

Test Fixtures (conftest.py)

Test Directory Structure

Running Backend Tests Locally

Coverage

Frontend Tests

Unit Tests (Jest)

End-to-End Tests (Playwright)

Test Suites and Tags

Test Sharding

CI Test Execution

Backend CI Matrix

Full Test Execution Flow

Test Configuration Summary

pyproject.toml pytest settings (root)

src/backend/base/pyproject.toml pytest settings

Blocking I/O Detection

Load Testing

Ruff Overrides for Test Files

On this page

Testing Strategy

Overview

Backend Tests

Framework and Configuration

Test Markers

Pytest Plugins Used

Test Fixtures (conftest.py)

Test Directory Structure

Running Backend Tests Locally

Coverage

Frontend Tests

Unit Tests (Jest)

End-to-End Tests (Playwright)

Test Suites and Tags

Test Sharding

CI Test Execution

Backend CI Matrix

Full Test Execution Flow

Test Configuration Summary

pyproject.toml pytest settings (root)

src/backend/base/pyproject.toml pytest settings

Blocking I/O Detection

Load Testing

Ruff Overrides for Test Files

On this page

Test Fixtures (`conftest.py`)

`pyproject.toml` pytest settings (root)

`src/backend/base/pyproject.toml` pytest settings

Test Fixtures (`conftest.py`)

`pyproject.toml` pytest settings (root)

`src/backend/base/pyproject.toml` pytest settings