swiftsync-protocol

Wed Jun 24 2026

The session started from one question: in IBD today, what is the actually expensive part? My first guess was block validation, the signature and script checks. That is wrong, and the correction is the whole foundation.

The bottleneck is the UTXO set, not signature checks

Here is what a normal node does on every single spend: look up the coin in the UTXO set, then delete it. That is the part SwiftSync kills.

Why it hurts. The UTXO set is about 180 million outputs and several gigabytes, too big for RAM, so it sits on disk in a database keyed by outpoint. From there, three things go wrong:

• Outpoints are basically random, so every spend reads a random spot on disk, and random reads are far slower than sequential ones.
• Deletes are not free. The database marks the coin dead and reclaims the space later in heavy background passes.
• The set is shared state updated in strict block order, so it cannot be spread across CPU cores the way signature checks can.

Trading random access for ordered, batched work is one of the most common wins in systems engineering, and SwiftSync is one instance of it.

Check a guess instead of building the set

The set you would have built obeys one invariant. Every output ever created either survives into the final UTXO set or gets spent:

$\text{created outputs} -\text{surviving UTXO} = \text{spent inputs}$

SwiftSync verifies this by content. It keeps two running aggregates:

$\text{Agg} = H(\text{Coin}_0) + H(\text{Coin}_1) + \dots \pmod{2^{256}}$

Created coins (minus the survivors) feed one aggregate, spent coins feed the other. At the end, check Agg_outputs == Agg_inputs. Two properties make this work, doing different jobs:

• Addition mod $2^{256}$ is commutative and associative. Partial sums on different threads, in any order, combine to the same total. That is where the parallelism comes from. Parallelism is the payoff, not the root idea.
• The hash is deterministic. A coin hashed at creation gives the exact same value when hashed at spend, so the two cancel. That is what lets a "do the sums balance" check stand in for actually matching every creation to its spend.

The coin: five fields, each earned

The thing being hashed is a coin, carrying exactly five fields: outpoint, script, amount, coinbase flag, creation height. Three justify themselves: the outpoint is identity, the script and amount are what you check to authorize and balance the spend. The other two are about consensus rules enforced only at spend time:

• Coinbase flag and creation height enforce coinbase maturity: a coinbase output cannot be spent until 100 blocks after it was mined, checked as spend_height - creation_height >= 100. Neither fact lives in the spending input, so both travel inside the coin.
• Creation height also matters for ordinary coins via relative timelocks (BIP68 with OP_CHECKSEQUENCEVERIFY), which demand the spent coin be at least N blocks old.

But there is a bigger reason every field gets hashed. Under SwiftSync the spend-side coin bytes do not come from a UTXO set you maintain. They come from undo data handed to you by a peer who could be lying, and the aggregate equality is the only gate. So any field not inside the hash is a field the untrusted provider can forge without breaking the balance. Leave the amount out of the hash and they can inflate it freely. That is why all five ride along.

Where the spend-side bytes come from

At creation the full coin sits in the block you are processing, so you have everything. At spend time the input carries only a 36-byte outpoint (32-byte txid plus 4-byte index), so the script, amount, and height are missing. They come from undo data (Bitcoin Core's rev*.dat), downloaded alongside each block so the block stays self-contained and parallelism survives. Why it has to be undo data, and not the hintsfile or a lookup into the referenced block, is its own note: swiftsync-undo-data-self-contained-blocks.

The hintsfile: a compact list of survivors

The hintsfile answers one yes/no question per output: does this output survive into the UTXO set at height n? The naive design stores each survivor's 36-byte outpoint. For 180 million survivors that is $180 \times 36 = 6480$ MB, about 6.5 GB, absurd for something whose whole selling point is a fast bootstrap. The real file is around 119 MB at height 930,000. Two moves buy back that 30x-plus gap:

1. Positional handles. The client and the hintsfile author both walk every output in the same fixed creation order, without ever talking to each other. So a survivor can be named by its position in that order rather than its 36-byte outpoint, and the name shrinks to almost nothing. The hints are partitioned per block, each block carrying its own little encoding, so parallel random access still works. A single global index would not: block N's outputs would need the cumulative output count of every prior block, which you may not have downloaded yet.
2. Sparse indices, not a bitset. In a typical block almost every output eventually gets spent and only a handful survive. For a set that sparse, one bit per output is wasteful. Store the short sorted list of survivor indices instead. The scheme is Elias-Fano, which packs $n$ increasing indices in about $2n + n\log_2(m/n)$ bits, close to the information-theoretic floor, and needs no external library.

The salt: why it must be secret

Drop the salt and the hash is just public SHA-256(coin), with the only check being that two sums mod $2^{256}$ are equal. An attacker with some freedom over coin contents (a trivially-spendable output lets you pick its script) can hunt for two different multisets of coins whose sums collide. That is a generalized birthday attack.

The salt shuts it down, but only because it is secret. A public salt, even prefixed to every coin, buys nothing: the attacker just folds the known value into their own precomputation and searches as before. Secrecy is the property doing the work, not the prefixing. An attacker who cannot evaluate the hash function at all cannot run the offline search to begin with.

The spine

Why does Agg_outputs == Agg_inputs prove the UTXO set you assembled is right? When the two balance, they are the same multiset of spent coins described two ways: once as "everything created minus what the hints say survives", once as "everything actually consumed". A wrong hints file leaves some coin uncancelled on one side and the equality breaks. The match is by coin content, not count, so a forger would have to engineer colliding contents, which is exactly the door the secret salt slams shut.

One line to keep: SwiftSync trades building the UTXO set for checking a guess at it, and a secret hash-sum makes the guess impossible to fake.

What stayed shaky

• Elias-Fano internals: how it actually packs the monotone indices. Deferred on purpose, it is a self-contained algorithm.
• Generalized birthday attack mechanics: the k-list (Wagner) search itself, not just what it threatens.
• The BIP68 relative-timelock height detail, if I want it nailed down exactly.