Run an indexer

The indexer turns the blockchain into something you can query. It reads the ARIADNE application’s events, keeps them in a SQLite database, derives the tables a reader needs (articles, authors, reviews, comments, reputation…), fetches and checks the content from IPFS, and serves everything as the Indexer API. The chain is the truth; the database is a cache that can always be rebuilt.

Anyone can run one: to check ariadne.press (see Verify without trusting ariadne.press), to build a tool, or to serve a network. The source is projects/indexer, and its README.md is the detailed reference. Terms such as round, event and pin are explained in the Glossary.

Algorand application calls (ARC-28 logs)
  -> algokit-subscriber (watermark, catch-up)
  -> ARC-28 decoder built from the ARC-56 file        src/events.ts
  -> chain_events (append-only)                       src/db.ts
  -> projections, one transaction per round           src/projections.ts
  -> content checks (re-hash, header against chain)   src/content.ts
  -> REST API (read-only JSON)                        src/api.ts, src/queries.ts

One process serves one network and one application, with its own database.

Requirements

• Node 24 or later (engines in package.json; developed and accepted on Node 26.5.1, which the Docker image uses).
• No build step: Node runs the TypeScript sources directly. No native modules: the database is Node’s built-in node:sqlite.
• Network access to an Algorand node, and preferably an Algorand indexer for fast catch-up (both named in spec/networks.json).
• The repository: the indexer reads spec/networks.json and the contract’s ARC-56 file from it.

cd projects/indexer
npm ci
npm run check        # type check (tsc --noEmit)
npm test             # offline tests
npm run fmt:check    # formatting

Quick start on TestNet

cd projects/indexer
ARIADNE_NETWORK=testnet node src/main.ts serve

In PowerShell, set the variable first: $env:ARIADNE_NETWORK = "testnet"; node src/main.ts serve.

The indexer checks the node’s genesis hash, starts at the application’s creation round (startRound in the manifest), catches up through the Algorand indexer, and serves the API on http://127.0.0.1:3000:

curl http://127.0.0.1:3000/status

Without a Kubo node it reads content from the manifest’s IPFS gateway, verifying every block, and keeps no IPFS copy of its own. The database is data/testnet.sqlite inside projects/indexer unless you set ARIADNE_DB.

Configuration

The network entry of spec/networks.json gives the application id, the Algorand node and indexer, the IPFS gateway and the outside services (see Networks and addresses). Environment variables select the network and override or complete it. Numbers must be non-negative integers, or the indexer refuses to start.

Variable	Default	Meaning
ARIADNE_NETWORK	localnet	the key of spec/networks.json to serve
ARIADNE_NETWORKS_JSON	the repository’s spec/networks.json	another manifest file
ARIADNE_DB	data/<network>.sqlite	the database file; {appId} in it is replaced by the application id, so a new deployment starts with a new database
ARIADNE_HOST	127.0.0.1	the address the API listens on (the Docker image sets 0.0.0.0)
ARIADNE_PORT	3000	the API port
ARIADNE_POLL_SECONDS	2	pause between polls once caught up
ARIADNE_MAX_ROUNDS	500	rounds read from the node per poll while catching up
ARIADNE_ALGOD	the manifest’s algod	Algorand node
ARIADNE_ALGOD_TOKEN	the manifest’s algodToken	its API token
ARIADNE_APP_ID	the manifest’s appId	the application
ARIADNE_START_ROUND	the manifest’s startRound	where an empty database starts
ARIADNE_ALGO_INDEXER	the manifest’s algoIndexer	Algorand indexer for catch-up; empty: every round is read from the node
ARIADNE_IPFS_GATEWAY	the manifest’s ipfsGateway	trustless gateway used as content source; empty: none
ARIADNE_CONTENT_DIR	unset	a local content source: <dir>/<cid>/ holds an article folder, <dir>/<cid> a review or comment (tests, LocalNet)
ARIADNE_KUBO_API	unset	Kubo RPC address (for example http://kubo:5001): content source and second copy. Never expose this RPC publicly
ARIADNE_CONTENT_RECHECK_SECONDS	21600 (6 hours)	how often a content record is checked again
ARIADNE_CONTENT_PER_CYCLE	20	article CIDs checked per content cycle (review and comment texts: five times as many)
ARIADNE_CONTENT_CONCURRENCY	4	fetches at the same time (also pins at the same time)
ARIADNE_CONTENT_TIMEOUT_SECONDS	30	seconds before a fetch through Kubo gives up
ARIADNE_PINS_PER_CYCLE	20	pins attempted per pin cycle
ARIADNE_STALL_SECONDS	300	/health answers 503 after this long without a successful ingestion poll
ARIADNE_APP_URL	the manifest’s appUrl	public origin of the web app; ORCID and DOI records must link back under it
ARIADNE_ORCID_API	the manifest’s orcidApi	ORCID public API; empty: ORCID iDs are not checked
ARIADNE_ZENODO_API	the manifest’s zenodoApi	this network’s Zenodo
ARIADNE_DATACITE_API	the manifest’s dataciteApi	DataCite API; empty: DOIs other than Zenodo’s stay “declared”
ARIADNE_OPENALEX_API	https://api.openalex.org	OpenAlex, for verified ORCID iDs; empty turns it off
ARIADNE_ORCID_CLIENT_ID, ARIADNE_ORCID_CLIENT_SECRET	unset	ORCID /read-public credentials, for higher rate limits; without them the public API is used anonymously
ARIADNE_ORCID_TOKEN_URL	https://orcid.org/oauth/token	where the ORCID token is requested
ARIADNE_VERIFY_PER_CYCLE	10	ORCID iDs and DOIs checked per verification cycle, each
ARIADNE_VERIFY_RECHECK_SECONDS	604800 (7 days)	how often a settled ORCID or DOI result is checked again
ARIADNE_VERIFY_TIMEOUT_SECONDS	20	timeout of one request to an outside service

Commands

Run node src/main.ts <command> from projects/indexer (ingest, rebuild, serve and content also exist as npm scripts).

Command	What it does
serve	The normal way to run it: the API, the ingestion loop and every background loop.
ingest	Catches up with the chain once, without the API, and stops. ingest --follow keeps polling.
rebuild	Drops every projection table and rebuilds it from the stored chain_events (after a schema change or a fix). The result is identical to ingesting round by round.
content	Fetches and checks the article content that is due, once.
pins	Pins on the Kubo node what is due, once (needs ARIADNE_KUBO_API).
verify	Checks the declared ORCID iDs and DOIs that are due against ORCID, Zenodo, DataCite and OpenAlex, once.
reindex	Drops and rebuilds the search indexes from the database (serve keeps them current).
status	Prints the network, application, genesis, watermark, schema version and row counts as JSON.

What the indexer refuses

It stops with an error, rather than serving wrong data, when:

• the manifest’s appId is 0 (“deploy the contract first, or set ARIADNE_APP_ID”);
• the Algorand node reports a genesis hash different from the manifest’s;
• the database was built for another genesis or another application;
• the database schema is newer than the code.

Ingestion

Each poll asks for the application’s calls (and for the payment notes of public lists) after the last round stored, the watermark. With an Algorand indexer configured, a poll catches up by up to 100 000 rounds through it; otherwise every block is read from the node, ARIADNE_MAX_ROUNDS at a time, which needs an archival node. A new database starts just before startRound, so it never scans the years before ARIADNE existed.

Every round is one SQLite transaction: the new events, the projections and the watermark together. Storing an event twice changes nothing, so a crash or a restart simply resumes from the watermark. The projections apply the same rules as the contract’s reference implementation; an event stream that breaks them (which must never happen) aborts the round, the indexer logs it and keeps retrying, and /health turns 503 after ARIADNE_STALL_SECONDS.

Background loops

serve runs the ingestion loop and four background loops side by side, so a slow outside service never delays a new article. Each background loop pauses 10 seconds between cycles.

Loop	What it does
ingestion	Polls the chain (see above), every ARIADNE_POLL_SECONDS once caught up.
content	Checks the article CIDs that are due: never-checked ones first, newest article first, then re-checks, least recently checked first; up to ARIADNE_CONTENT_PER_CYCLE per cycle, ARIADNE_CONTENT_CONCURRENCY at a time. Also reads the text of new reviews and comments for search, keeping a text only when its bytes hash to its CID.
pins	With ARIADNE_KUBO_API, pins the second copy: every article CID checked as valid and every review and comment CID. Each pin may take 60 seconds; failures are retried after 1, 2, 4… minutes, at most a day.
verify	Checks declared ORCID iDs and DOIs, and asks OpenAlex for what the seal needs (see below).
search	Recomputes the citations between articles and the Thread Score, then the Peer-reviewed seal, then the search indexes, each only when something it depends on changed. Thread percentiles are also recomputed every day.

Content sources are tried in this order: ARIADNE_CONTENT_DIR, then Kubo (ARIADNE_KUBO_API), then the manifest’s IPFS gateway as a trustless gateway (/ipfs/<cid>?format=car, every block verified, the folder re-hashed with the canonical parameters). With no source at all, every article is unavailable: served with a warning, never hidden. The validation statuses are described in The article header.

Database

One SQLite file per network and application, in WAL mode (so -wal and -shm files sit beside it). The schema version is 10. Opening an older database migrates it in place; a newer one is refused.

Kind	Tables	Rebuilt by
Base tables, kept across rebuilds	meta, chain_events, watermark, content, pins, orcid_checks, doi_checks, post_texts, coauthor_checks	never: they are the inputs (events) or cached results of IPFS and outside services
Projections, from the events	fields, governance, articles, versions, coauthors, article_votes, reviews, review_votes, comments, comment_votes, mentions, flags, disputes, reputation, reputation_changes, claims, notifications, identities, article_dois, follows, lists, list_items	rebuild
Derived views	citations and article_impact (Thread Score); article_seals, review_qualifications, seal_state (the seal); search_articles, search_posts, search_state, search_posts_done (search)	the serve loop; reindex for search

chain_events holds every decoded event, keyed by transaction id and log position, with its round, name and fields. Every projection row points back to the event that produced it. Public lists are stored there too, as ListNote rows, so that rebuild replays them in chain order.

Nothing in the database needs a backup: delete it and the indexer rebuilds it from the chain (from startRound), then asks IPFS and the outside services again.

Outside services

Besides Algorand and IPFS, the indexer asks outside services about the declarations people make on chain. Their answers are cached in the base tables; no reputation is ever derived from them. Every request names ARIADNE in its User-Agent.

Service	Used for	Configured by
ORCID public API	Is a declared ORCID iD verified (its public record lists the person’s profile page)? The record’s names and public contact data	orcidApi / ARIADNE_ORCID_API; optional credentials
Zenodo records API	Is a declared Zenodo DOI published, and identical to the article page (and, for one version, its CID)?	the instance of the DOI’s prefix: 10.5281 on zenodo.org, 10.5072 on sandbox.zenodo.org (the network’s own zenodoApi when the prefix is its own)
DataCite	The same check for other DOIs	dataciteApi / ARIADNE_DATACITE_API
OpenAlex	Citation counts and works per field of verified ORCID iDs; works shared by a reviewer and an author (a conflict of interest for the seal)	ARIADNE_OPENALEX_API

Schedule: a settled result is checked again weekly (ARIADNE_VERIFY_RECHECK_SECONDS); during the two days after a declaration that is not verified yet, every ten minutes; a failed request is retried after 1, 2, 4… minutes, at most a day, and the last result is kept meanwhile. Reviewer-author pairs are checked on OpenAlex every 30 days.

Turning them off

To stop	Set	Effect
ORCID	ARIADNE_ORCID_API= (empty)	declared iDs stay pending
DataCite	ARIADNE_DATACITE_API= (empty)	DOIs that are not Zenodo’s stay declared
OpenAlex	ARIADNE_OPENALEX_API= (empty)	no citation counts; the seal’s OpenAlex path to expertise and its shared-works conflict check are not available
Every ORCID and DOI lookup	ARIADNE_APP_URL= (empty)	ORCID iDs are not checked, DOIs are declared (malformed ones invalid), and links to article pages are not recognised as citations

Zenodo DOIs are checked at the Zenodo instance of their prefix even when ARIADNE_ZENODO_API is empty; only an empty ARIADNE_APP_URL stops those lookups. An indexer with outside services turned off computes the seal and the citations from less information, so its results can differ from those of ariadne.press.

Docker

deploy/indexer.Dockerfile builds an image from the repository root: Node 26.5.1 (pinned by digest), production dependencies only, the sources, the ARC-56 file and spec/. It runs node src/main.ts serve as the node user, with ARIADNE_HOST=0.0.0.0, ARIADNE_PORT=3000 and a volume at /data.

docker build -f deploy/indexer.Dockerfile -t ariadne-indexer .
docker run -d --name ariadne-indexer-testnet \
  -e ARIADNE_NETWORK=testnet \
  -e "ARIADNE_DB=/data/testnet-{appId}.sqlite" \
  -v ariadne-indexer-testnet:/data \
  -v "$PWD/spec:/app/spec:ro" \
  -p 127.0.0.1:3000:3000 \
  ariadne-indexer

Mounting spec/ read-only, as the hosting stack does, lets a change to the manifest take effect with a restart instead of a rebuild. Other commands run in the same image:

docker exec ariadne-indexer-testnet node src/main.ts status

The hosting stack adds a Kubo node (ARIADNE_KUBO_API=http://kubo:5001) and a Docker health check that reads /health every 60 seconds; see Host the full stack.

Logs

The indexer writes one line per event worth knowing: polls that found calls or caught up, content checked, pins, ORCID and DOI results, errors, and API requests that failed with a 5xx status. A reverse proxy in front of it keeps the access log.

LocalNet

For development, start the repository’s LocalNet and deploy the contract (see Networks and addresses), then:

cd projects/indexer
ARIADNE_CONTENT_DIR=/path/to/content node src/main.ts serve

ARIADNE_NETWORK defaults to localnet. Point ARIADNE_CONTENT_DIR at the same folder the web app uses for LocalNet (its own ARIADNE_CONTENT_DIR): the web app’s development pin target unpacks each published folder there, and the indexer validates it from there. The LocalNet acceptance test catches up from round 0 and compares the projections with every Article and CoAuthor box: populate the chain with the contract suite (poetry run pytest tests/localnet in projects/contracts), then run npm run test:localnet.