Streams Implementation

Relevant source files

• src/bun.js/bindings/BunProcess.cpp
• src/bun.js/bindings/BunProcess.h
• src/bun.js/bindings/c-bindings.cpp
• src/bun.js/bindings/helpers.cpp
• src/bun.js/modules/BunObjectModule.h
• src/bun.js/modules/ObjectModule.h
• src/bun.js/node/node_fs.zig
• src/bun.js/node/node_fs_binding.zig
• src/bun.js/node/types.zig
• src/js/builtins/ImportMetaObject.ts
• src/js/builtins/ProcessObjectInternals.ts
• src/js/builtins/ReadableStreamInternals.ts
• src/js/node/fs.promises.ts
• src/js/node/http.ts
• src/js/node/stream.ts
• src/main.zig
• src/output.zig
• src/sys.zig
• src/windows.zig
• test/cli/run/require-cache-bug-5188.js
• test/cli/run/require-cache.test.ts
• test/js/node/fs/fs.test.ts
• test/js/node/http/node-http.test.ts
• test/js/node/process/process.test.js
• test/js/node/stream/node-stream.test.js
• test/js/node/test/parallel/test-process-setsourcemapsenabled.js
• test/js/node/test/parallel/test-resource-usage.js
• test/regression/issue/26377.test.ts

Purpose and Scope

This document describes Bun's implementation of Node.js-compatible streams (node:stream) and their integration with WHATWG Streams. It covers the four core stream types (Readable, Writable, Duplex, Transform), process stdio stream initialization, file system streams, and the bridging layer between Node.js streams and native WHATWG Streams.

For information about the WHATWG Streams API (ReadableStream, WritableStream, TransformStream), see 5.6 Streams API. For HTTP-specific streaming behavior, see 8.3 HTTP Module. For file system APIs that use streams, see 8.2 File System APIs.

Architecture Overview

Bun's stream implementation operates on three layers: Node.js Stream API (the compatibility layer), Internal Stream Implementation (Bun's native implementation), and WHATWG Streams (the underlying primitive). The Node.js stream types inherit from EventEmitter and provide the familiar .pipe(), .on('data'), and .write() interfaces, while internally delegating to either native implementations or WHATWG Stream primitives.

Stream Type Hierarchy

Diagram: Node.js Stream Type Hierarchy and WHATWG Stream Integration

Sources: test/js/node/stream/node-stream.test.js1-67 src/js/node/stream.ts1-9

Stream Module Organization

Module Path	Purpose	Key Exports
node:stream	Main stream exports	Readable, Writable, Duplex, Transform, PassThrough, pipeline, finished
internal/stream	Internal stream implementation	Core stream classes and utilities
internal/streams/end-of-stream	Stream completion detection	eos (end-of-stream helper)
node:fs	File system streams	createReadStream, createWriteStream, ReadStream, WriteStream
node:tty	TTY streams	ReadStream, WriteStream
node:http	HTTP streams	IncomingMessage, ServerResponse, ClientRequest, OutgoingMessage

Sources: src/js/node/stream.ts1-9 src/js/node/http.ts1-72

Process Stdio Streams

Process stdio streams (process.stdin, process.stdout, process.stderr) are initialized during process startup and bridge between native file descriptors and Node.js stream interfaces.

Stdio Stream Initialization Flow

Diagram: Process Stdio Stream Initialization Pipeline

Sources: src/js/builtins/ProcessObjectInternals.ts33-91 src/js/builtins/ProcessObjectInternals.ts93-279

Stdin Implementation (process.stdin)

The stdin stream is implemented using getStdinStream() which bridges Bun.stdin.stream() (a WHATWG ReadableStream) to a Node.js-compatible stream.

Key implementation details:

Aspect	Implementation
Base class	tty.ReadStream (if TTY) or fs.ReadStream (if file/pipe/socket)
Native source	Bun.stdin.stream() provides WHATWG ReadableStream
Reader management	own()/disown() control reader lock and ref counting
Backpressure	Pauses native stream when Node.js stream pauses
Event bridge	Translates WHATWG stream events to Node.js events ('data', 'end', 'error')

The own() function acquires a reader lock on the WHATWG stream and refs it to keep the process alive, while disown() releases the lock and unrefs:

own() -> reader = native.getReader() -> source.updateRef(true) -> source.setFlowing(true)
disown() -> reader.releaseLock() -> source.updateRef(false) -> source.setFlowing(false)

Sources: src/js/builtins/ProcessObjectInternals.ts93-279

Stdout/Stderr Implementation

Stdout and stderr are write-only streams created by getStdioWriteStream():

Diagram: Stdout/Stderr Stream Creation

The key distinction between TTY and non-TTY streams:

• TTY streams: Use tty.WriteStream, have readable=true (for duplex compatibility), emit 'SIGWINCH' events on terminal resize, implement _refreshSize() method
• File streams: Use fs.WriteStream, have readable=false, include Symbol.asyncIterator only for pipes/sockets (not regular files)

Both override _destroy() to prevent actual closing of stdio file descriptors (fd 1 and 2), as closing these would break further output.

Sources: src/js/builtins/ProcessObjectInternals.ts33-91

Platform-Specific Stdio Initialization

Windows

On Windows, stdio initialization involves:

1. Console mode setup: UTF-8 code page, virtual terminal processing
2. Handle caching: Stdio handles cached in windows_cached_stdin/stdout/stderr
3. TTY detection: GetConsoleMode() determines if handle is a console
4. Mode flags: Set ENABLE_VIRTUAL_TERMINAL_PROCESSING for ANSI escape codes

Diagram: Windows Stdio Initialization

Sources: src/output.zig133-219

POSIX

On POSIX systems, the process is simpler:

1. Detect TTY: isatty() system call
2. Set flags: Store in bun_stdio_tty array
3. Terminal size: Query with ioctl(TIOCGWINSZ)

Sources: src/output.zig1-230

File System Streams

File system streams (fs.createReadStream, fs.createWriteStream) provide streaming access to files. They inherit from fs.ReadStream and fs.WriteStream respectively.

fs.ReadStream Architecture

Diagram: fs.ReadStream Lifecycle

Key ReadStream features:

Feature	Implementation	Code Reference
Lazy open	File opened on first read if not already open	N/A (internal)
Range reads	start and end options specify byte range	N/A (internal)
Backpressure	Respects highWaterMark, pauses when buffer full	N/A (internal)
Encoding	Optional encoding for automatic string conversion	N/A (internal)
Auto-close	autoClose option (default true) closes fd on end	N/A (internal)

Sources: test/js/node/fs/fs.test.ts259-326 test/js/node/stream/node-stream.test.js46-62

fs.WriteStream Architecture

Diagram: fs.WriteStream Lifecycle

Key WriteStream features:

Feature	Implementation
Append mode	flags: 'a' opens in append mode
Position tracking	Automatically tracks write position
Buffering	Internal buffering up to highWaterMark
Drain events	Emits 'drain' when buffer empties
Atomic writes	Single fs.write() call per chunk

Sources: test/js/node/fs/fs.test.ts258-326 src/js/builtins/ProcessObjectInternals.ts33-91

Fast Path Optimization

Bun implements a "fast path" for fs streams that bypasses some Node.js compatibility layers when possible. The fast path is accessed via the kWriteStreamFastPath symbol:

const underlyingSink = stream[require("internal/fs/streams").kWriteStreamFastPath];

This provides direct access to the native write implementation for better performance.

Sources: src/js/builtins/ProcessObjectInternals.ts88-90

HTTP Streams

HTTP request and response bodies in Bun use Node.js stream interfaces for compatibility.

HTTP Request Streaming (IncomingMessage)

Diagram: HTTP Request Streaming Flow

Request streaming example from tests:

server.on('request', (req, res) => {
  req.on('data', chunk => {
    res.write(chunk); // Echo back
  });
  req.on('end', () => {
    res.end();
  });
});

Sources: test/js/node/http/node-http.test.ts82-106 src/js/node/http.ts4-7

HTTP Response Streaming (ServerResponse)

Diagram: HTTP Response Streaming with Backpressure

The response stream properly implements backpressure: write() returns false when the internal buffer exceeds the high water mark, signaling to the producer to pause. The stream emits 'drain' when the buffer is flushed, signaling it's safe to resume writing.

Sources: test/js/node/http/node-http.test.ts82-135 src/js/node/http.ts1-72

OutgoingMessage Base Class

Both ServerResponse and ClientRequest inherit from OutgoingMessage, which provides:

• Header management (setHeader, getHeader, removeHeader)
• Chunked transfer encoding support
• Trailer support
• Write buffering
• Connection management

Sources: src/js/node/http.ts5-6

Stream Utilities and Helpers

pipeline()

The pipeline() utility connects multiple streams together with proper error handling:

pipeline(source, transform1, transform2, destination, (err) => {
  if (err) console.error('Pipeline failed:', err);
  else console.log('Pipeline succeeded');
});

Pipeline automatically:

• Pipes streams in sequence
• Propagates errors to all streams
• Cleans up on completion
• Calls the callback exactly once

finished() / eos

The finished() (alias eos for "end-of-stream") utility detects when a stream has completed:

finished(stream, (err) => {
  if (err) console.error('Stream error:', err);
  else console.log('Stream finished');
});

It properly handles:

• 'finish' event for writables
• 'end' event for readables
• 'close' event as fallback
• 'error' event for failures
• Premature closes

Sources: src/js/node/stream.ts1-9

Bridge to WHATWG Streams

Bun bridges between Node.js streams and WHATWG Streams in both directions.

Node.js Readable → WHATWG ReadableStream

Diagram: Node.js Readable to WHATWG ReadableStream Bridging

The adapter:

1. Listens to 'data' events and enqueues chunks to the WHATWG stream controller
2. Listens to 'end' to close the stream
3. Listens to 'error' to error the stream
4. Implements backpressure by pausing the Node.js readable when the WHATWG stream is full

WHATWG ReadableStream → Node.js Readable

The reverse direction is used for process.stdin:

Diagram: WHATWG ReadableStream to Node.js Readable Bridging

The bridge implementation in getStdinStream():

1. Acquires a reader from the WHATWG stream
2. Implements _read() to call reader.read() asynchronously
3. Pushes chunks to the Node.js readable as they arrive
4. Manages reader lock lifecycle with own() and disown()
5. Implements proper pause/resume behavior

Sources: src/js/builtins/ProcessObjectInternals.ts93-279 src/js/builtins/ReadableStreamInternals.ts29-44

Writable Stream Bridging

For writable streams, the bridge works similarly but in reverse:

Node.js Writable → WHATWG WritableStream:

• Implements WritableStreamUnderlyingSink
• Forwards write() calls to sink.write()
• Forwards end() to sink.close()
• Forwards backpressure signals

WHATWG WritableStream → Node.js Writable:

• Acquires a writer with getWriter()
• Implements _write() to call writer.write()
• Implements _final() to call writer.close()
• Manages writer lock lifecycle

Stream Type Conversion Table

From	To	Method	Implementation Location
Node.js Readable	WHATWG ReadableStream	Readable.toWeb()	Internal stream utilities
WHATWG ReadableStream	Node.js Readable	Readable.fromWeb()	Internal stream utilities
Node.js Writable	WHATWG WritableStream	Writable.toWeb()	Internal stream utilities
WHATWG WritableStream	Node.js Writable	Writable.fromWeb()	Internal stream utilities

Sources: src/js/builtins/ReadableStreamInternals.ts1-25

Error Handling and Resource Management

Stream Error Propagation

Diagram: Stream Error Handling Flow

Best practices for error handling:

• Always attach error handlers to streams
• Use pipeline() for automatic error propagation
• Call destroy(err) to properly clean up errored streams
• Check for backpressure (write() returns false) to prevent memory leaks

Sources: test/js/node/stream/node-stream.test.js1-67

Resource Cleanup

Streams should be properly closed to avoid resource leaks:

Stream Type	Cleanup Method	Automatic Cleanup
fs.ReadStream	stream.close() or stream.destroy()	Yes, if autoClose: true
fs.WriteStream	stream.end() or stream.destroy()	Yes, if autoClose: true
http.IncomingMessage	Automatic on request end	Yes
http.ServerResponse	res.end()	Yes
process.stdin/stdout/stderr	Never close (special case)	N/A

For process stdio streams, the _destroy() method is overridden to prevent actual closing:

stream._destroy = function(err, cb) {
  cb(err);
  this._undestroy();
  // Don't actually close fd 0, 1, or 2
};

Sources: src/js/builtins/ProcessObjectInternals.ts72-83

Performance Considerations

Buffering and Backpressure

Proper backpressure handling is critical for memory efficiency:

Diagram: Backpressure Flow

High water mark defaults:

Stream Type	Default highWaterMark	Unit
Readable (object mode)	16	objects
Readable (buffer mode)	16384 (16KB)	bytes
Writable (object mode)	16	objects
Writable (buffer mode)	16384 (16KB)	bytes

Zero-Copy Optimizations

Bun implements several zero-copy optimizations:

1. Direct buffer passing: Buffers are passed by reference when possible, avoiding copies
2. Fast path writes: The kWriteStreamFastPath symbol provides direct native writes
3. Platform-specific optimizations: Uses copy_file_range() on Linux, sendfile() where available
4. Memory mapping: Large files may be memory-mapped instead of read in chunks

Sources: src/bun.js/node/node_fs.zig1-4 src/js/builtins/ProcessObjectInternals.ts88-90

Stream Performance Tips

Scenario	Recommendation	Reason
Large file reads	Use createReadStream()	Avoids loading entire file into memory
Sequential writes	Use createWriteStream() with backpressure	Prevents memory exhaustion
HTTP body forwarding	Use .pipe()	Automatic backpressure handling
Transform operations	Use Transform stream	Built-in buffering and error handling
Binary data	Avoid encoding option	Prevents unnecessary string conversions

Testing and Validation

Bun's stream implementation is validated against Node.js behavior through extensive tests:

Test coverage areas:

• Basic stream operations (read, write, pipe)
• Backpressure and flow control
• Error handling and propagation
• Resource cleanup (fd closing)
• Event emission order
• Edge cases (empty streams, large data, slow consumers)
• Platform-specific behavior (Windows vs POSIX)
• Integration with HTTP, fs, and other modules

Sources: test/js/node/stream/node-stream.test.js1-667 test/js/node/fs/fs.test.ts1-1227 test/js/node/http/node-http.test.ts1-727

Common Patterns and Examples

Piping Multiple Streams

// Example: Read file → compress → write file
const fs = require('fs');
const zlib = require('zlib');

fs.createReadStream('input.txt')
  .pipe(zlib.createGzip())
  .pipe(fs.createWriteStream('output.txt.gz'))
  .on('finish', () => console.log('Done'));

Transform Stream Implementation

const { Transform } = require('stream');

class UpperCase extends Transform {
  _transform(chunk, encoding, callback) {
    this.push(chunk.toString().toUpperCase());
    callback();
  }
}

// Usage
process.stdin.pipe(new UpperCase()).pipe(process.stdout);

Custom Readable Stream

const { Readable } = require('stream');

class CounterStream extends Readable {
  constructor(max) {
    super();
    this.max = max;
    this.current = 0;
  }
  
  _read() {
    if (this.current < this.max) {
      this.push(String(this.current++));
    } else {
      this.push(null); // Signal EOF
    }
  }
}

Sources: test/js/node/stream/node-stream.test.js8-67

---

This implementation provides comprehensive Node.js stream compatibility while leveraging WHATWG Streams as the underlying primitive, allowing Bun to offer both familiar Node.js APIs and modern web-standard stream interfaces.