This page covers how uBlock Origin (MV2) discovers, fetches, caches, differentially updates, and compiles filter lists into its filtering engines. The primary files involved are src/js/assets.js, assets/assets.json, src/js/diff-updater.js, and src/js/storage.js.
This page does not cover:
assets.json for the purpose of adding new lists — see 11.2assets.jsonassets/assets.json is the central registry of all known filter lists and internal assets. It is the authoritative source of truth for what lists exist, where to fetch them, and how often to update them.
At runtime, the content of assets.json is loaded into assetSourceRegistry inside src/js/assets.js. A development variant, assets/assets.dev.json, is used when the build flavor includes devbuild src/js/background.js211-213
Each key in assets.json is an asset key (e.g., "ublock-filters", "easylist") and maps to an object with the following fields:
| Field | Type | Description |
|---|---|---|
content | string | "internal" (system asset) or "filters" (filter list) |
group | string | UI category: "default", "ads", "privacy", "malware", "annoyances", "regions" |
title | string | Human-readable name shown in the dashboard |
contentURL | string | string[] | Primary fetch URL(s); local bundled path may appear last |
cdnURLs | string[] | CDN mirror URLs used when not in favorOrigin mode |
patchURLs | string[] | Base URLs for fetching diff patch files |
updateAfter | number | Days before the asset is considered stale (default: 7) |
off | boolean | If true, the list is not enabled by default |
supportURL | string | Link to the list's issue tracker |
parent | string | null | Used to group related lists in the dashboard |
tags | string | Space-separated tags (e.g., "ads privacy") |
lang | string | Language codes for region-specific lists |
Example entry (abbreviated from assets/assets.json38-59):
Sources: assets/assets.json1-996 assets/assets.dev.json1-60
assets.js Modulesrc/js/assets.js exports a single object io (imported as io throughout the codebase via import io from './assets.js'). It manages the full lifecycle of any asset: fetching, caching, updating, and notifying observers.
Key internal state:
| Variable | Role |
|---|---|
assetSourceRegistry | In-memory copy of assets.json; populated by assets.updateStart() |
assetCacheRegistry | Per-asset cache metadata: writeTime, expires, diffPath, diffExpires |
remoteServerFriendly | Boolean flag; when true, suppresses cache-busting query params |
observers | Array of registered observer functions notified of update events |
Exported methods:
| Method | Description |
|---|---|
assets.fetch(url, options) | XHR-based fetch with timeout and progress tracking |
assets.fetchText(url) | Fetches a URL as plain text; discards HTML responses |
assets.fetchFilterList(mainlistURL) | Fetches a filter list, resolving !#include sub-lists |
assets.get(assetKey, options) | Gets an asset from cache or network |
assets.put(assetKey, content) | Stores content in the asset cache |
assets.metadata() | Returns metadata for all known assets |
assets.updateStart() | Begins the periodic auto-update cycle |
assets.updateStop() | Stops the update cycle |
assets.getUpdateAges(details) | Returns the age of specified assets |
assets.addObserver(fn) / assets.removeObserver(fn) | Subscribe/unsubscribe from asset events |
Sources: src/js/assets.js1-200
Diagram: URL Resolution Chain in getContentURLs
Sources: src/js/assets.js181-213
The function getContentURLs src/js/assets.js181-213 builds the ordered list of URLs to try:
contentURL entries (string or array) form the base list.favorLocal is set, local paths (not matching reIsExternalPath) are sorted first.cdnURLs are shuffled randomly (to distribute CDN load) and prepended or appended depending on favorLocal.favorOrigin is set, CDN URLs are omitted entirely.Cache busting: assets.fetchText appends a _=<token> query parameter to external URLs to avoid stale browser caches src/js/assets.js340-357 The token rotates hourly by default (Math.floor(Date.now() / 3600000) % 13), or can be forced to rotate every second via hiddenSettings.updateAssetBypassBrowserCache.
Each filter list file may contain metadata in its first 1024 bytes as comment lines of the form ! Field: value or # Field: value. The function extractMetadataFromList src/js/assets.js67-94 reads these fields.
Fields used by the asset system:
| Filter List Header | Field in Code | Purpose |
|---|---|---|
! Last-Modified: | lastModified | Determines if a remote copy is newer than cache |
! Expires: | expires | Overrides updateAfter from assets.json |
! Diff-Expires: | diffExpires | Expiry interval for the diff patch path |
! Diff-Path: | diffPath | Relative path to the current diff patch file |
The presence of a valid Diff-Path header marks an asset as diff-updatable, checked by isDiffUpdatableAsset src/js/assets.js150-157
Assets are stored and retrieved through cacheStorage (imported from src/js/cachestorage.js), which is a multi-tier storage layer (IndexedDB, Cache API, or sessionStorage). See 4.3 for details on the storage backend.
The assetCacheRegistry holds metadata for all cached assets. Key fields per entry:
| Field | Meaning |
|---|---|
writeTime | Unix timestamp of when content was last written to cache |
expires | Expiry in days, extracted from list header or assets.json |
diffPath | Last known Diff-Path value from the list's metadata |
diffExpires | Expiry interval for diff patches |
remoteURL | The URL from which the cached content was last fetched |
resourceTime | Last-Modified time from the remote server at last fetch |
getWriteTime(assetKey) and getUpdateAfterTime(assetKey, diff) use this registry to determine staleness src/js/assets.js125-142
For filter lists that publish frequent, incremental changes, uBlock Origin supports differential (diff) updates to avoid downloading entire lists on every update cycle. This mechanism is implemented in src/js/diff-updater.js.
Diagram: Differential Update Flow
Sources: src/js/diff-updater.js1-280
A patch file is a plain text file containing one or more diff blocks. Each block begins with a diff line specifying the asset name, line count, and SHA-1 checksum:
diff name:filters.min.txt lines:42 checksum:a1b2c3d4e5
a100 3
! New filter line 1
! New filter line 2
! New filter line 3
d250 1
The diff format is RCS-style: a<line> <count> to add lines, d<line> <count> to delete lines src/js/diff-updater.js107-144
diff-updater.js| Function | Description |
|---|---|
parsePatch(patch) | Parses a patch file into a Map<name, {diff, checksum, lines}> |
applyPatch(text, diff) | Applies an RCS-style diff to a string, returns patched text |
applyPatchAndValidate(assetDetails, diffDetails) | Applies patch and validates result with SHA-1 via crypto.subtle.digest |
fetchPatchDetails(assetDetails) | Fetches and caches the patch file from CDNs (deduplicates concurrent fetches via patches Map) |
fetchAndApplyAllPatches(assetDetails) | Orchestrates the full differential update; sets assetDetails.status |
The patches Map src/js/diff-updater.js26 deduplicates concurrent fetches of the same patch file across multiple assets that share a CDN patch bundle.
When filter lists need to be loaded (at startup or after an update), the pipeline is orchestrated by µb.loadFilterLists() defined in src/js/storage.js.
Diagram: Filter List Loading Pipeline
Sources: src/js/storage.js452-480 src/js/start.js1-450
A selfie is a serialized binary snapshot of all compiled filtering engines. It allows the extension to skip the expensive parse-and-compile step on subsequent startups.
loadFilterLists() call, µb.selfieManager serializes the compiled state of all filtering engines into cacheStorage.systemSettings.compiledMagic or systemSettings.selfieMagic changes, both defined in src/js/background.js src/js/background.js183-185 Currently at value 60.onCacheSettingsReady() src/js/start.js272-286 checks the magic numbers against stored values. If they match, µb.selfieManager.load() is attempted instead of full recompilation.!#include Directive Supportassets.fetchFilterList(mainlistURL) src/js/assets.js392 handles sublists referenced by !#include <relative-path> directives. It:
!#include directives (only those not gated by !#if that evaluated to false).sublistURLs Set.µb.selectedFilterLists (an array of asset keys) is the canonical record of which filter lists are active. It is persisted via vAPI.storage as the selectedFilterLists key.
Diagram: selectedFilterLists State Management
Sources: src/js/storage.js452-558
Key methods on µb:
| Method | Description |
|---|---|
loadSelectedFilterLists() | Reads selectedFilterLists from vAPI.storage; on first launch, auto-selects defaults via autoSelectRegionalFilterLists() |
saveSelectedFilterLists(newKeys, append) | Persists the set; purges cache for removed keys |
applyFilterListSelection(details) | Processes toSelect, toRemove, and toImport from dashboard UI; handles URL-to-assetKey mapping for imported lists |
listKeysFromCustomFilterLists(raw) | Parses a newline-separated URL list into asset keys; filters out entries in µb.badLists |
The auto-update cycle is initiated by assets.updateStart() and driven by vAPI.alarms or vAPI.defer. During an update:
assets.metadata() is called to get the current state of all assets.µb.selectedFilterLists, the update age is checked against getUpdateAfterTime(assetKey).isDiffUpdatableAsset(cachedContent) is true, a diff update is attempted via diff-updater.js.contentURL / cdnURLs.assets.addObserver() are notified with topic "after-asset-updated"."assets-update-completed".µb observer in storage.js intercepts "assets-update-completed" to trigger µb.loadFilterLists() if any filter list changed.Relevant hidden settings (from src/js/background.js):
| Setting | Default | Effect |
|---|---|---|
autoUpdatePeriod | 1 | Hours between auto-update checks |
autoUpdateDelayAfterLaunch | 37 | Seconds to wait after launch before first update |
autoUpdateAssetFetchPeriod | 5 | Seconds between individual asset fetches during update |
manualUpdateAssetFetchPeriod | 500 | Milliseconds between fetches during manual update |
differentialUpdate | true | Enable/disable the diff update system |
Sources: src/js/background.js44-92 src/js/assets.js125-157
assets.js implements a simple observer pattern src/js/assets.js217-239 Modules register callbacks via assets.addObserver(fn). The observer function receives a (topic, details) call:
| Topic | When fired |
|---|---|
"before-assets-updated" | Before the update cycle starts |
"after-asset-updated" | After a single asset is successfully updated |
"assets-update-completed" | After the entire update cycle finishes |
"asset-content-changed" | When a user-facing asset (e.g., user filters) is written |
The primary observer is registered in src/js/storage.js and responds to "assets-update-completed" by calling µb.loadFilterLists() if any filter list content changed.
The µb.isTrustedList(assetKey) method src/js/storage.js416-443 determines whether a filter list is trusted to use advanced filter features (e.g., trusted scriptlets). A list is trusted if:
user-filters) and userFiltersTrusted is set, orhiddenSettings.trustedListPrefixes (default: "ublock-").This trust status is checked by the messaging handler for getAssetContent src/js/messaging.js97-106 which sets result.trustedSource accordingly before returning content to the dashboard editor.
Refresh this wiki
This wiki was recently refreshed. Please wait 2 days to refresh again.