This section covers two topics that fall outside the core runtime systems: the dig development tooling for benchmarking the Static Net Filtering Engine (SNFE), and the assets.json configuration schema that governs all filter list fetching and updating.
For information about how the SNFE works internally, see Static Network Filtering Engine. For how filter lists are fetched and compiled at runtime, see Filter List Management and Updates. For the MV3 build process and ruleset generation, see MV3 Ruleset Generation Pipeline.
dig HarnessThe dig platform is a standalone Node.js environment that builds a copy of the SNFE outside the browser extension, allowing it to be benchmarked and interrogated from the command line.
The dig tooling lives in platform/dig/ and is built using the dig and dig-snfe Makefile targets defined in Makefile58-65
make dig # builds dist/build/uBlock0.dig and installs npm deps
make dig-snfe # runs the SNFE benchmark
make dig-snfe record compare maxcost
The dig target copies shared source files and the platform/npm/ adapter into dist/build/uBlock0.dig, then runs npm install in that directory to pull the scaling-palm-tree request dataset.
Diagram: dig tool build flow
Sources: Makefile58-65 platform/dig/package.json1-28 platform/dig/snfe.js1-40
snfe.js Bench Harnessplatform/dig/snfe.js is the primary benchmark script. It loads the SNFE with several real filter lists, then runs every request from the scaling-palm-tree dataset through engine.matchRequest().
Runtime flags are read from process.argv platform/dig/snfe.js31-39:
| Flag | Effect |
|---|---|
record | Writes results to data/snfe.json |
compare | Diffs current results against data/snfe.json from a prior run |
maxcost | Writes the 1000 most expensive matches to data/snfe.maxcost.json |
medcost | Writes the median 1000 matches by cost |
mincost | Writes the 1000 cheapest matches |
modifiers | Benchmarks matchAndFetchModifiers instead of matchRequest |
wasm | Enables WASM code paths before the benchmark |
Diagram: snfe.js control flow
Sources: platform/dig/snfe.js318-377
The matchRequests function platform/dig/snfe.js84-158 iterates all requests, calls engine.matchRequest(details), and counts blocked, unblocked, and not-blocked results. Per-request timing uses process.hrtime.bigint().
The compare function platform/dig/snfe.js160-185 loads a previously recorded data/snfe.json and reports which request indices have changed their block decision (r field) between runs.
The matchRequestModifiers function platform/dig/snfe.js189-274 separately benchmarks modifier matching: csp, redirect-rule, and removeparam are each tested with engine.matchAndFetchModifiers().
Request type mapping is defined at platform/dig/snfe.js42-61: puppeteer resource type strings (e.g., document, xhr, fetch) are translated into WebRequest type strings (e.g., sub_frame, xmlhttprequest) before being passed to the engine.
All filter lists and internal assets are registered in assets/assets.json. This file is read by assets.js at startup and controls what gets fetched, from where, how often, and whether differential updates are available.
A parallel file, assets/assets.dev.json, provides a development variant. It is structurally identical but points assets.json's own contentURL to the development version rather than the production file assets/assets.dev.json3-14
assets.json SchemaEach key in assets.json is an asset identifier (e.g., "easylist", "ublock-filters"). The value object has the following fields:
| Field | Type | Description |
|---|---|---|
content | string | "filters", "internal", or similar. Controls how the asset is processed after fetch. |
group | string | Logical group for display in the dashboard (e.g., "default", "ads", "privacy", "annoyances", "regions"). |
group2 | string | Secondary grouping (e.g., "cookies", "social"). |
parent | string | Display-level parent label for grouped entries. |
title | string | Human-readable name shown in the filter lists pane. |
contentURL | string or array | Primary fetch URL(s), tried in order. Local paths (assets/...) are used as bundled fallbacks. |
cdnURLs | array | CDN mirrors, tried in shuffled order. |
patchURLs | array | Base URL prefixes used by the diff-updater to construct patch file URLs. |
supportURL | string | Link to the upstream list's issue tracker or home page. |
off | boolean | If true, the list is disabled by default. Omitting means enabled by default. |
preferred | boolean | Hints that this list is a preferred choice among alternatives. |
updateAfter | number | Minimum hours before the asset should be re-checked for updates. |
tags | string | Space-separated search tags for the filter list UI. |
lang | string | Space-separated BCP 47 language codes for regional lists. |
ua | string | Platform hint (e.g., "mobile") for conditional display. |
instructionURL | string | Optional link shown alongside entries that need context (e.g., badware risks). |
Diagram: assets.json entry → asset lifecycle
Sources: assets/assets.json1-60 src/js/diff-updater.js1-285
Lists that declare patchURLs are eligible for differential (patch-based) updates. The mechanism is implemented in src/js/diff-updater.js.
The patch URL is constructed by appending a filename derived from the current date and version to a patchURLs base. The parsePatch function src/js/diff-updater.js68-105 reads the multi-asset patch file format, which contains one diff header block per named list. The applyPatch function src/js/diff-updater.js107-144 applies a line-based RCS-style diff. After application, applyPatchAndValidate src/js/diff-updater.js156-181 computes a SHA-1 digest of the result and compares it to the checksum embedded in the patch header.
The module runs as a web worker. Its self.onmessage handler src/js/diff-updater.js270-283 accepts { what: 'update', ... } messages and posts the updated asset details back when complete.
Status values returned in assetDetails.status:
| Status | Meaning |
|---|---|
updated | Patch applied and validated successfully |
nodiff | Patch file was fetched but contained no entry for this list |
nopatch | No patch file found at any patchURLs |
nopatch-yet | Patch time has not arrived yet (based on filename date) |
needtext | Patch was found but the current text of the list was not provided |
baddiff | Patch application produced an invalid result |
badchecksum | Post-patch SHA-1 did not match expected |
Sources: src/js/diff-updater.js223-264
When building for development, assets.dev.json is substituted for assets.json. The key difference is in the assets.json entry itself: the development file points contentURL at the assets.dev.json remote path so that subsequent self-updates pull the dev registry rather than production assets/assets.dev.json3-14
All other list entries in assets.dev.json are identical to their production counterparts.
| Page | Contents |
|---|---|
| SNFE Benchmarking and Development Tools | Full documentation of the dig tool: build steps, flag reference, output files, and the npm package for external SNFE consumers. |
| Asset and Filter List Configuration | Detailed assets.json schema reference, the diff-update patch mechanism, assets.dev.json usage, and how to register a new filter list. |
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.