This page documents the dig development tool located in platform/dig/, which provides a Node.js harness for benchmarking the Static Net Filtering Engine (SNFE) against a real-world request dataset. It also covers the @gorhill/ubo-core npm package in platform/npm/, which exposes the SNFE for external consumers.
For documentation of the SNFE internals themselves (filter classes, the filterMatch pipeline, compiled filter serialization), see page 3.1 For the filter list loading and compilation pipeline that feeds data into both tools, see page 3.4
The dig tool is a standalone Node.js package at platform/dig/. It is not distributed or published; it exists solely for development-time performance measurement and correctness comparison of the SNFE.
Its entry point is platform/dig/snfe.js. It imports StaticNetFilteringEngine and enableWASM from the built output at ./index.js (produced by the make-dig.sh build script).
The Makefile exposes two relevant targets:
| Target | Command | Effect |
|---|---|---|
dig | make dig | Runs tools/make-dig.sh, then npm install in the output dir |
dig-snfe | make dig-snfe [flags] | Runs npm run snfe inside dist/build/uBlock0.dig |
The build output lands in dist/build/uBlock0.dig. The npm install step fetches the scaling-palm-tree dev dependency (a GitHub-hosted request dataset) declared in platform/dig/package.json.
platform/dig/package.json → devDependencies:
"scaling-palm-tree": "github:mjethani/scaling-palm-tree#15cf1ab"
This package supplies the JSON request corpus (node_modules/scaling-palm-tree/requests.json) used as input to every benchmark run.
Sources: Makefile58-65 platform/dig/package.json1-28
Diagram: dig benchmark data flow
Sources: platform/dig/snfe.js318-377 platform/dig/snfe.js31-39
Flags are passed as positional arguments to npm run snfe, or via make dig-snfe flag1 flag2 .... The Makefile includes no-op pseudo-targets for shell auto-completion convenience (Makefile173-187).
| Flag | Constant | Effect |
|---|---|---|
record | RECORD | Writes per-request results to data/snfe.json (or data/snfe.modifiers.json) |
compare | COMPARE | Loads a previously recorded data/snfe.json and diffs it against the current run; writes data/snfe.diffs.json |
maxcost | MAXCOST | Writes the 1000 most expensive requests to data/snfe.maxcost.json |
medcost | MEDCOST | Writes 1000 requests around the median cost to data/snfe.medcost.json |
mincost | MINCOST | Writes the 1000 cheapest requests to data/snfe.mincost.json |
modifiers | MODIFIERS | Switches from basic match benchmarking to modifier matching (csp, redirect-rule, removeparam) |
wasm | WASM | Calls enableWASM() before running; enables WASM-accelerated code paths in the engine |
Any combination of flags is valid. If none of record, compare, maxcost, medcost, mincost is set, the constant NEED_RESULTS is false and per-request timing is skipped, reducing benchmark overhead.
Sources: platform/dig/snfe.js31-39
matchRequests)The matchRequests function platform/dig/snfe.js84-158 is the primary benchmark path.
Steps:
cpt (puppeteer resource type) field to a WebRequest type string using WEBREQUEST_OPTIONS platform/dig/snfe.js42-61engine.matchRequest(details) for each request.r === 0 → not blockedr === 1 → blockedr === 2 → unblocked (exception rule matched)NEED_RESULTS is true, records per-request timing using process.hrtime.bigint() and captures the matching filter via engine.toLogData().raw.Console output format:
Filter lists parsed-compiled-loaded in X ms
Matched N requests in X ms
Not blocked: N requests
Blocked: N requests
Unblocked: N requests
Average: X µs per request
Sources: platform/dig/snfe.js84-158 platform/dig/snfe.js65-71
matchRequestModifiers)The matchRequestModifiers function platform/dig/snfe.js189-274 benchmarks the three modifier types that apply after the basic block/allow decision.
Diagram: matchRequestModifiers decision logic
Each modifier type is checked independently. A single request may trigger multiple modifier checks. The modifiedCount accumulates a count of requests where at least one modifier was found.
Sources: platform/dig/snfe.js189-274
The compare function platform/dig/snfe.js160-185 and compareModifiers function platform/dig/snfe.js276-314 load a previously record-ed JSON file from data/ and diff it against the current run.
Workflow for regression testing:
record flag on a known-good engine version → writes data/snfe.json.compare flag → reads data/snfe.json, runs again, writes data/snfe.diffs.json containing only requests where the block/allow result changed.Each diff entry records:
type, url, originURL — request identitybefore.r, before.f, before.t — old result, filter text, timingafter.r, after.f, after.t — new result, filter text, timingSources: platform/dig/snfe.js160-185 platform/dig/snfe.js276-314
The bench() function platform/dig/snfe.js318-377 passes a hardcoded list of filter lists to engine.useLists():
| List Name | Asset Path |
|---|---|
filters | assets/ublock/filters.min.txt |
badware | assets/ublock/badware.txt |
privacy | assets/ublock/privacy.min.txt |
quick-fixes.txt | assets/ublock/quick-fixes.txt |
unbreak.txt | assets/ublock/unbreak.txt |
easylist | assets/thirdparties/easylist/easylist.txt |
easyprivacy | assets/thirdparties/easylist/easyprivacy.txt |
PGL | assets/thirdparties/pgl.yoyo.org/as/serverlist |
urlhaus | assets/thirdparties/urlhaus-filter/urlhaus-filter-online.txt |
These are read from the assets/ directory of the built dist/build/uBlock0.dig artifact, which is populated by the build scripts from the uAssets submodule.
A dry run platform/dig/snfe.js359-366 iterates over every 8th request before the timed run, giving the JS engine time to JIT-compile hot paths.
Sources: platform/dig/snfe.js335-357 platform/dig/snfe.js359-366
@gorhill/ubo-core)Separate from the dig tool, the repository also produces a publishable npm package at platform/npm/. This package exports the StaticNetFilteringEngine for use by external Node.js consumers — the same class that platform/dig/snfe.js imports from ./index.js.
The dig tool's index.js is the locally built version of this same interface. The npm package's build is driven by the npm Makefile target:
dist/build/uBlock0.npm: tools/make-nodejs.sh $(sources) $(platform) $(assets)
tools/make-npm.sh
The published package name is @gorhill/ubo-core. For MV3 builds, the npm module has no direct relationship to the dig benchmarking tooling; it is a separate output artifact.
Sources: Makefile44-47
Diagram: Code entities in the dig/npm ecosystem
Sources: Makefile44-66 platform/dig/snfe.js23-27 platform/dig/package.json1-28
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.