This page covers the data structures and algorithms that store terminal text, character attributes, and associated metadata. The core of this system is the TextBuffer class and its per-row ROW struct, both defined under src/buffer/out/. The text buffer is used by both the Windows Terminal engine and by conhost.
For how VT sequences write into this buffer, see VT Sequence Processing. For how the contents of this buffer are rendered to the screen, see Renderer Architecture. For the higher-level console screen buffer (SCREEN_INFORMATION) that wraps TextBuffer in conhost, see Console Host Architecture.
| Type | File | Role |
|---|---|---|
TextBuffer | src/buffer/out/textBuffer.hpp | Top-level buffer: owns rows, cursor, hyperlinks, marks |
ROW | src/buffer/out/Row.hpp | One row: text chars, column offsets, attributes |
RowAttributes | src/buffer/out/Row.hpp | RLE-encoded TextAttribute sequence for a row |
TextAttribute | src/buffer/out/TextAttribute.hpp | SGR state: colors, bold, underline, hyperlink ID, etc. |
til::small_rle | src/inc/til/rle.h | Run-length encoding container used for RowAttributes |
TextBufferCellIterator | src/buffer/out/textBufferCellIterator.hpp | Bidirectional iterator over buffer cells |
TextBufferTextIterator | src/buffer/out/textBufferTextIterator.hpp | Text-only iterator layered on TextBufferCellIterator |
CharToColumnMapper | src/buffer/out/Row.hpp | Maps a char offset inside a ROW's text to a terminal column |
ImageSlice | src/buffer/out/ImageSlice.hpp | Optional image content attached to a ROW |
ScrollMark / MarkExtents | src/buffer/out/Marks.hpp | Shell-integration prompt/command marks |
Sources: src/buffer/out/textBuffer.hpp1-424 src/buffer/out/Row.hpp1-200 src/buffer/out/lib/bufferout.vcxproj1-54
Ownership diagram:
Sources: src/buffer/out/textBuffer.hpp68-423 src/buffer/out/Row.hpp98-200
TextBuffer uses a single VirtualAlloc reservation for all rows. Memory is initially only MEM_RESERVEd. Pages are MEM_COMMITted lazily as rows are first accessed, in batches of _commitReadAheadRowCount (128) rows at a time. This keeps the private working set small (around 2 MB at startup).
Each row slot occupies a stride _bufferRowStride calculated at construction:
[ ROW struct ] CalculateRowSize() — aligned to 16 bytes
[ _charsBuffer ] CalculateCharsBufferSize(w) — width × sizeof(wchar_t), aligned to 16 bytes
[ _charOffsets buffer ] CalculateCharOffsetsBufferSize(w) — (width+1) × sizeof(uint16_t), aligned to 16 bytes
Diagram — Physical memory layout of the arena:
Key fields controlling this:
| Field | Purpose |
|---|---|
_buffer | Base of the virtual address reservation |
_bufferEnd | One past end of the reservation |
_commitWatermark | First uncommitted byte; rows below this are ready to use |
_bufferRowStride | Bytes per row slot |
_bufferOffsetChars | Byte offset within a slot to _charsBuffer |
_bufferOffsetCharOffsets | Byte offset within a slot to _charOffsets buffer |
The scratchpad row lives at physical slot 0. Regular rows start at slot 1. GetScratchpadRow() always resets and returns slot 0. _getRow(y) maps logical y → physical (_firstRow + y) % _height + 1.
Sources: src/buffer/out/textBuffer.cpp86-208 src/buffer/out/textBuffer.hpp356-414 src/buffer/out/Row.hpp111-122
The row array is used as a circular buffer. _firstRow holds the physical index of logical row 0 (the top of the visible area). Scrolling down by one line does not move any data; it simply:
GetMutableRowByOffset(0).Reset(...))_firstRow (wrapping at _height)Mapping logical index → physical slot:
physical_offset = (_firstRow + y) % _height
↓ +1 for scratchpad offset
physical_slot = physical_offset + 1
GetRowByOffset(y) is the read-only accessor; GetMutableRowByOffset(y) increments _lastMutationId before returning, allowing consumers to detect mutations cheaply.
Sources: src/buffer/out/textBuffer.cpp717-735 src/buffer/out/textBuffer.cpp191-207 src/buffer/out/textBuffer.hpp125-134
Each ROW represents one terminal row. Its character storage uses a two-part design to support grapheme clusters wider than one wchar_t (e.g. emoji, surrogate pairs):
| Member | Type | Description |
|---|---|---|
_charsBuffer | wchar_t* | Points into the arena allocation; holds _columnCount chars |
_charsHeap | unique_ptr<wchar_t[]> | Heap fallback when row needs more chars than columns (e.g. emoji sequences) |
_chars | std::span<wchar_t> | Points to whichever of the above is active |
_charOffsets | std::span<uint16_t> | _columnCount + 1 entries; _charOffsets[col] = char index for column col |
_attr | RowAttributes | Run-length encoded TextAttribute per column |
_columnCount | uint16_t | Number of terminal columns |
_wrapForced | bool | True if line was wrapped by content (not a hard line break) |
_doubleBytePadded | bool | True if a wide glyph was split at end of line |
_lineRendition | LineRendition | SingleWidth, DoubleWidth, DoubleHeightTop/Bottom |
_imageSlice | ImageSlice* | Optional image content (e.g. sixel); null when absent |
_promptData | optional<...> | Shell-integration prompt/output state |
_charOffsets EncodingWide glyphs (e.g. CJK characters) occupy two columns but one or more wchar_t code units. _charOffsets bridges terminal columns to char positions.
The high bit (CharOffsetsTrailer = 0x8000) marks a column as the trailing half of a wide glyph. The lower 15 bits (CharOffsetsMask = 0x7fff) hold the actual char index.
Example: A 4-column row containing a, a wide glyph WW, and b:
Column: 0 1 2 3
Char: 'a' 'W' '-' 'b' (2 wchar_t for 'WW')
_charOffsets: 0 1 0x8001 3 (col 2 has trailer bit set)
CharToColumnMapper performs the inverse: given a pointer into _chars, it binary-searches _charOffsets to find the owning column.
Sources: src/buffer/out/Row.hpp237-330 src/buffer/out/Row.cpp83-158
RowAttributes is defined as:
til::small_rle (in src/inc/til/rle.h) is a run-length encoding container. It stores a vector of rle_pair<TextAttribute, uint16_t> — each pair holds an attribute value and a run length (number of columns sharing that attribute).
Most rows have only a single attribute run (the common case: same colors everywhere), which is why the inline buffer size is 2. When a row has many different attribute runs, small_rle spills to the heap.
Diagram — RLE attribute storage for a row:
Key RowAttributes / til::small_rle operations used by ROW:
| Operation | Method |
|---|---|
| Overwrite a range with one attribute | ReplaceAttributes(beginIndex, endIndex, newAttr) |
| Set attribute from column to end | SetAttrToEnd(columnBegin, attr) |
| Read by column | GetAttrByColumn(column) |
| Forward iterate (all columns) | AttrBegin() / AttrEnd() |
| Extract a sub-range | _attr.slice(...) |
| Replace a sub-range | _attr.replace(...) |
Sources: src/inc/til/rle.h1-400 src/buffer/out/Row.hpp20-21 src/buffer/out/Row.hpp152-158
Diagram — Writing path from VT dispatch to TextBuffer:
| Function | Signature summary | Behavior |
|---|---|---|
TextBuffer::Write | (OutputCellIterator, point, optional<bool> wrap) | Writes cells starting at point, advancing across rows |
TextBuffer::WriteLine | (OutputCellIterator, point, optional<bool> wrap, optional<CoordType> limitRight) | Writes one line only |
TextBuffer::Replace | (row, attributes, RowWriteState&) | Overwrites text and attributes; updates wrap/dirty info |
TextBuffer::Insert | (row, attributes, RowWriteState&) | Inserts text, shifting remainder right |
TextBuffer::FillRect | (rect, fill wstring_view, attributes) | Fills a rectangular region with a repeated character |
RowWriteState is a struct passed by reference that carries both input parameters and output results:
RowWriteState {
text // IN/OUT: text to write, updated to remaining after call
columnBegin // IN: first column to write
columnLimit // IN: exclusive column limit
columnEnd // OUT: one past last column written
columnBeginDirty / columnEndDirty // OUT: actual dirty range (may extend past written range for wide-glyph splits)
}
ROW::_init() uses SIMD (AVX2 or SSE2/NEON) to initialize _charsBuffer with spaces and _charOffsets with 0, 1, 2, .... This is the hot path for scrollback and screen-clearing operations.
Sources: src/buffer/out/textBuffer.cpp527-625 src/buffer/out/textBuffer.cpp633-709 src/buffer/out/Row.hpp29-73 src/buffer/out/Row.cpp241-354
TextBuffer exposes two iterator types for reading:
| Iterator | Created via | Scope |
|---|---|---|
TextBufferCellIterator | GetCellDataAt(point) | Reads OutputCellView (char + attribute) per cell |
TextBufferTextIterator | GetTextDataAt(point) | Text-only wrapper over TextBufferCellIterator |
Both can be optionally bounded to a Viewport, in which case iteration stops at the edge rather than wrapping. GetCellLineDataAt() creates a TextBufferCellIterator bounded to a single row.
TextBufferCellIterator supports operator++ / operator-- and can report how many cells were advanced (GetCellDistance()).
TextBuffer::GetCellDistance(from, to) counts glyph positions between two points by incrementing a TextBufferCellIterator until it reaches the target.
Sources: src/buffer/out/textBuffer.hpp93-100 src/buffer/out/textBuffer.cpp288-377
TextBuffer::CopyRequest is a configuration struct for extracting text:
| Field | Purpose |
|---|---|
beg, end | Inclusive start/end coordinates |
blockSelection | True → rectangular selection |
trimTrailingWhitespace | Strip trailing spaces from each row |
includeLineBreak | Add \r\n between rows |
formatWrappedRows | Apply line-break formatting to wrapped rows |
bufferCoordinates | Whether coords are in buffer or screen space |
CopyRequest::FromConfig() computes the correct combination of flags from high-level intent (single-line mode, block selection, trim setting).
Text extraction methods:
| Method | Output |
|---|---|
GetPlainText(CopyRequest) | Plain wstring |
GetWithControlSequences(CopyRequest) | wstring with VT SGR sequences preserving colors |
GenHTML(CopyRequest, ...) | std::string HTML with inline styles |
GenRTF(CopyRequest, ...) | std::string RTF document |
SerializeTo(HANDLE) | Writes buffer to a handle (used for session restore) |
Search operates on the entire buffer or a row range. It uses ICU regex internally via UTextAdapter (src/buffer/out/UTextAdapter.h), which adapts ROW text storage to ICU's UText interface without copying all text first.
Sources: src/buffer/out/textBuffer.hpp202-302 src/buffer/out/textBuffer.hpp301-303
When the terminal is resized to a different width, Reflow copies text from oldBuffer into newBuffer (which is sized to the new dimensions), rewrapping lines at the new column count. Wrapped lines (rows where WasWrapForced() == true) are treated as continuations of the previous logical line and reflowed together.
PositionInformation is an output struct that tracks how the mutable viewport top and visible viewport top shift after reflow, so the terminal engine can scroll the view to approximately the same content.
Sources: src/buffer/out/textBuffer.hpp299-300 src/buffer/out/textBuffer.cpp1-50
TextBuffer maintains a bidirectional map for OSC 8 hyperlinks:
| Member | Type | Purpose |
|---|---|---|
_hyperlinkMap | unordered_map<uint16_t, wstring> | ID → URI |
_hyperlinkCustomIdMap | unordered_map<wstring, uint16_t> | Custom string ID → numeric ID |
_currentHyperlinkId | uint16_t | Next ID to assign |
The numeric hyperlink ID is stored inside TextAttribute for each cell. When a row scrolls out of the buffer, _PruneHyperlinks() is called from IncrementCircularBuffer() to remove entries whose IDs are no longer referenced by any remaining row.
Relevant methods:
| Method | Purpose |
|---|---|
GetHyperlinkId(uri, id) | Returns (or creates) a numeric ID for a URI + optional custom ID |
AddHyperlinkToMap(uri, id) | Inserts a URI → ID mapping |
GetHyperlinkUriFromId(id) | Looks up URI by numeric ID |
RemoveHyperlinkFromMap(id) | Deletes an entry |
CopyHyperlinkMaps(other) | Copies maps from another buffer (used during reflow) |
Sources: src/buffer/out/textBuffer.hpp195-201 src/buffer/out/textBuffer.hpp351-353
TextBuffer supports shell integration annotations (prompt, command, output regions). Each ROW can carry optional ScrollbarData (used to render colored marks on the scrollbar) and _promptData for tracking prompt state.
Buffer-level methods:
| Method | Purpose |
|---|---|
StartPrompt() | Marks current row as start of a prompt |
StartCommand() | Marks transition from prompt to command input |
StartOutput() | Marks start of command output |
EndCurrentCommand(error) | Closes command region with optional exit code |
GetMarkRows() | Returns rows with scroll marks |
GetMarkExtents(limit) | Returns extent info for all mark regions |
ClearMarksInRange(start, end) | Removes marks in a coordinate range |
CurrentCommand() | Extracts current command text |
Commands() | Returns all completed commands in the buffer |
ScrollMark and MarkExtents types are defined in src/buffer/out/Marks.hpp.
Sources: src/buffer/out/textBuffer.hpp304-316 src/buffer/out/Row.hpp180-183
This diagram shows the principal callers of TextBuffer and what they use it for.
Sources: src/buffer/out/textBuffer.hpp68-423 src/buffer/out/textBuffer.cpp1-50 src/buffer/out/lib/bufferout.vcxproj1-54
Refresh this wiki
This wiki was recently refreshed. Please wait 4 days to refresh again.