This document explains how to execute the RealWorld API test suite against any backend implementation. The test suite uses Newman (Postman's CLI runner) to validate API compliance with the OpenAPI specification.
For information about the test collection structure and assertion patterns, see Postman Collection Architecture. For details on what the tests validate and coverage metrics, see Test Coverage & Validation.
The test infrastructure uses Newman CLI to execute the Postman collection api/Conduit.postman_collection.json against a target API endpoint. The testing system supports two execution modes:
Both methods execute the same test collection but differ in how they manage the target API server lifecycle.
Sources: api/run-api-tests.sh1-20 Makefile63-77
The test suite accepts environment variables to configure test execution and generate unique test data. These variables are passed to Newman as global variables via the --global-var flag.
| Variable | Default | Description | Used By |
|---|---|---|---|
APIURL | https://api.realworld.io/api | Base URL of the API to test | run-api-tests.sh6 |
USERNAME | u{timestamp} | Username for test user registration | run-api-tests.sh7 |
EMAIL | {USERNAME}@mail.com | Email for test user registration | run-api-tests.sh8 |
PASSWORD | password | Password for test user authentication | run-api-tests.sh9 |
DELAY_REQUEST | 500 | Milliseconds delay between requests | run-api-tests.sh11 |
JWT_SECRET | (required for server) | Secret for JWT token generation | Makefile58-68 |
APIURL: Must include the full base path including /api suffix. Examples:
http://localhost:3000/apihttps://api.realworld.io/apihttps://my-backend.example.com/apiUSERNAME: Dynamically generated using date +%s by default to ensure uniqueness across test runs. The format u{timestamp} prevents collisions when running concurrent tests.
DELAY_REQUEST: Controls rate limiting to prevent overwhelming the API server. Lower values (e.g., 50) speed up tests but may cause failures on slower backends. Higher values (e.g., 500) are more reliable but slower.
JWT_SECRET: Only required when running the reference implementation server, not when executing tests. The Makefile uses a fixed secret dxLmhnE0pRY2+vUlu+i5Pxh8LTxLBTgBWdp82W74mMs= for consistent local testing.
Sources: api/run-api-tests.sh6-11 Makefile58-68
The api/run-api-tests.sh script provides direct control over test execution. It wraps the Newman CLI with environment variable configuration.
The script api/run-api-tests.sh13-19 constructs this command with default values and allows override via environment variables.
To test the reference implementation running on localhost:3000:
Adjust DELAY_REQUEST for faster execution against local servers:
Sources: api/run-api-tests.sh1-20 api/README.md5-9
To test any remote API implementation:
For remote servers, use the default DELAY_REQUEST=500 to avoid rate limiting issues.
Override default credentials for specific test scenarios:
Sources: api/run-api-tests.sh6-11
The Makefile provides high-level targets that orchestrate server management and test execution. These targets abstract the complexity of starting servers and handling process lifecycle.
Sources: Makefile57-77
The reference-implementation-test-with-postman target handles the complete lifecycle:
This target Makefile66-77:
JWT_SECRET in background (npm -C apps/api run dev &)SERVER_PID=$$!)trap "kill $$SERVER_PID 2>/dev/null || true" EXIT)sleep 0.3)kill -0 "$$SERVER_PID")reference-implementation-test-with-postman-and-already-launched-serverThe test configuration uses DELAY_REQUEST=50 and APIURL=http://localhost:3000/api as specified in Makefile64
For iterative development, manually start the server and run tests separately:
The reference-implementation-run-for-postman target Makefile57-58 sets the required JWT_SECRET and starts the development server.
The reference-implementation-test-with-postman-and-already-launched-server target Makefile63-64 only executes the test script without server management.
Sources: Makefile57-77
The running-processes-clean target Makefile82-87 forcefully terminates all Nitro development server processes:
This searches for processes matching the full path /apps/api/node_modules/.bin/nitro dev and kills them with SIGKILL. This is necessary because simply killing the parent shell process does not always terminate the underlying Nitro server.
Sources: Makefile82-87
The following diagram shows the detailed execution flow when running tests, including stateful test orchestration.
Sources: Makefile66-77 api/run-api-tests.sh13-19 api/Conduit.postman_collection.json120-147
For rapid iteration during backend development:
For automated testing in continuous integration:
This is used in .github/workflows/api.yaml for automated testing on every commit.
To validate a new RealWorld implementation running on a different server:
Enable verbose Newman output by passing additional arguments:
Or use Newman's reporters for detailed output:
The script api/run-api-tests.sh19 passes through additional arguments ("$@") to Newman, allowing full access to Newman CLI options.
Sources: api/run-api-tests.sh19 Makefile63-77 api/README.md1-12
The test suite uses dynamically generated usernames based on the current timestamp api/run-api-tests.sh7 to avoid conflicts:
This generates usernames like u1704067200, ensuring each test run creates a fresh user. The test collection maintains state across requests using Postman's global variables:
This stateful execution allows the test suite to validate complete workflows including resource creation, retrieval, update, and deletion.
Sources: api/run-api-tests.sh7 api/Conduit.postman_collection.json120-147
Refresh this wiki