How a camera frame becomes a URL.

01Grayscale

src/detect/grayscale.ts

A camera frame arrives as RGBA bytes. QR codes carry no color information, so the first move is to collapse each pixel to its brightness using the classic BT.601 weights: 0.299·R + 0.587·G + 0.114·B, computed in integer arithmetic as (77·R + 150·G + 29·B) >> 8. One pass, one byte per pixel. Every later stage now works on a quarter of the data.

02Adaptive binarization

src/detect/binarizer.ts

Deciding which pixels count as "dark" is where real-world photos go to die. A single global threshold fails the moment lighting varies across the frame. One half of the symbol sits in glare, the other in shadow, and no single cutoff separates both.

So every pixel is judged against its surroundings: a square window about a tenth of the frame wide. The window's mean brightness comes from a summed-area table built once per frame, which makes it four lookups no matter how big the window is. A pixel counts as dark when it sits clearly below its window's mean. That alone would hollow out ink regions larger than the window (their interior is the local mean), so a second cut keeps anything far below the global Otsu threshold, the classic histogram split, dark as well. Frames too small for meaningful windows use Otsu alone.

The output is a packed bit image. The tryInverted option (on by default) simply flips it and runs detection again. That is the whole cost of supporting light-on-dark codes.

03Finding the finder patterns

src/detect/finder.ts · spec §7.3.2

The three big squares in a QR code's corners are engineered to be findable: any straight line through a finder pattern's center crosses dark, light, dark, light, dark runs in the ratio 1:1:3:1:1, whatever the symbol's rotation or size. That one property is the entire location strategy.

Every row of the bit image is run-length encoded, and each five-run window is tested against the ratio, allowing a summed deviation of up to a quarter of the window's width. A row hit is only a candidate. It gets cross-checked by walking the same ratio vertically through the proposed center, which also refines the center estimate. A single noise pixel on the exact center column would veto an otherwise perfect pattern, so failed walks retry up to two pixels to either side before giving up.

Confirmed centers from many rows are clustered. Repeated confirmation raises a pattern's count, which later serves as a confidence signal. There is deliberately no diagonal cross-check (a common extra filter): one bad pixel on the diagonal would veto a well-confirmed center, and the geometry scoring in the next stage filters false positives more robustly.

04Choosing the triple

src/detect/finder.ts · rankTriples

A frame may hold more finder-like blobs than the three we need: decoys in busy backgrounds, or six and more real patterns when several codes share the frame. Every combination of three candidates is scored against the right-isosceles corner layout of a real symbol. Module sizes must agree (a spread beyond half the mean rejects the triple outright). The two legs must be similar in length, and long enough that the patterns don't overlap. The diagonal must be close to √2 legs, and the corner angle close to 90°.

The best-scoring triple is then ordered geometrically: the corner opposite the longest side is top-left, and the sign of a cross product tells top-right from bottom-left. That is how a code scans identically at any rotation. Lower-ranked triples are not thrown away. tryHarder iterates through them so a decoy cannot mask the real symbol, and decodeAll() walks them all, retiring each pattern as its symbol decodes. That is how several codes per frame partition cleanly.

05Module size, version, and the alignment pattern

src/detect/detector.ts · src/detect/alignment.ts · spec §7.3.5

How many modules wide is this symbol? The finder's edge profile is measured from each center along the actual directions toward the other finders. The core→ring→rim transitions sit at exactly 1.5, 2.5, and 3.5 modules, and measuring along the pattern's own axes keeps the estimate honest under rotation, where a horizontal ruler would over-read by 1/cos θ. Dividing center distances by the module size gives a provisional dimension, snapped to the only values the spec allows (4k+17, i.e. 21…177 modules). Estimates that snap badly reject the triple.

Symbols of version 2 and up embed a small 1:1:1:1:1 alignment pattern near the bottom-right corner. Its expected position is predicted from the finder parallelogram, then searched for in the bit image. Found, it becomes the fourth anchor that captures perspective tilt. Not found (hidden under a thumb, say), the parallelogram's completed fourth corner stands in: less accurate, still usually decodable.

06The homography

src/detect/perspective.ts

A phone never looks at a poster straight-on. The mapping between the tilted symbol in the photo and its ideal square grid is a plane projective transform: a 3×3 homography, the same math that texture-maps a quad in graphics. Four point pairs pin it down: three finder centers plus the alignment pattern (or the completed parallelogram corner).

cam2qr computes it the textbook way. The four correspondences linearize into eight equations in the homography's eight unknowns, solved by Gaussian elimination with partial pivoting. And then it never warps the image. Instead, the transform runs backwards: for each module of the ideal grid, project its center into the photo and read one pixel. Sampling beats warping on both speed and accuracy. There are no resampling artifacts, and the cost scales with the symbol's module count, not its pixel area.

07Sampling the grid

src/detect/detector.ts · sampleModules

One projected read per module center (offset +0.5 to hit centers, not corners) turns the photo into an exact N×N bit matrix. Sub-pixel overshoot at the outermost ring of modules is clamped to the image edge. Anything worse means the geometry was wrong, and the triple is rejected rather than decoded into garbage. From here on there are no pixels. Only bits.

08Format information: the 15 most protected bits

src/core/format.ts · src/core/bch.ts · spec §8.9

Before the payload can be read, the decoder must learn two things the encoder chose: the error-correction level, and which of eight mask patterns was applied. These five bits ride in a 15-bit field protected by a BCH(15,5) code that survives up to three flipped bits. They have to be bulletproof, because misreading them corrupts everything downstream. The field is XOR'd with a fixed pattern (so an all-zero choice never produces an all-blank field) and written twice around the finder patterns. cam2qr decodes the two copies jointly, scoring every candidate against the better-preserved reading, so one copy mangled toward a different valid codeword can never shadow a clean one. Versions 7 and up also carry an 18-bit BCH(18,6) version field in two copies, decoded the same way and cross-checked against stage 05's size estimate. A legible disagreement rejects the fit rather than decoding garbage.

09Unmasking

src/core/mask.ts · spec §8.8.1

Encoders XOR the data region with one of eight checkerboard-like formulas, chosen to break up solid areas and accidental finder-lookalikes that would confuse scanners. Each formula is a one-line predicate over the module's coordinates ((i+j) % 2 === 0 is mask 000). Decoding is simply applying the announced formula again, because XOR is its own inverse. Function patterns (finders, timing lines, alignment, the format field itself) are never masked, so the unmasker walks the data region only.

10The zigzag walk and de-interleaving

src/core/codewords.ts · spec §8.7.3

Data bits live along a serpentine path. Starting at the bottom-right corner, the reader climbs a two-module-wide column upward, steps left, descends the next pair, and so on, detouring around every function pattern it meets. Eight bits at a time become codewords.

Those codewords are not in payload order. Larger symbols split their data into multiple error-correction blocks and interleave them codeword by codeword, so a coffee stain wipes out a little of every block instead of all of one. The decoder reverses the interleave using the block structure table for this version and EC level.

11Reed–Solomon error correction

src/core/reed-solomon.ts · src/core/gf256.ts · spec §8.5

This is the stage that makes QR codes work in the real world: up to roughly 30% of a symbol (at level H) can be wrong, and the payload still comes back exact. Every block is a Reed–Solomon codeword over GF(256), an arithmetic where bytes are polynomials modulo the primitive polynomial 0x11D, built once into log/antilog tables so that multiplying is two lookups.

Decoding runs the textbook sequence. Compute syndromes (all zero means the block is clean, the common fast path). Run Berlekamp–Massey over the syndromes to build the shortest error-locator polynomial Λ that explains them. Find Λ's roots by Chien search; each root names a damaged byte position. Compute each error's magnitude with Forney's formula, and XOR the fixes in. If more positions are damaged than the block can correct, the math fails loudly and the decode is rejected. Never silently wrong. The number of corrected codewords rides on every result as ecc.codewordsCorrected: a free damage gauge.

12Segments: bits become text

src/core/segments.ts · spec §8.4

The corrected bytes are a bitstream of mode-tagged segments, each squeezing its character class close to its information-theoretic floor. Numeric packs three digits into 10 bits. Alphanumeric packs two characters into 11. Byte is raw octets, decoded as strict UTF-8 with an ISO-8859-1 fallback (the de-facto standard). Kanji packs a Shift-JIS pair into 13 bits. And ECI switches the charset for whatever follows, using the AIM assignment table.

Three header-like modes round out the spec. FNC1 marks GS1 data (where % in alphanumeric text is an escape for the GS separator). Structured Append declares "this is symbol i of N" so the scanner can reassemble split payloads, parity-checked before anything is emitted. And the terminator ends the stream. Finally the decoded text is classified into result.content: URL, WiFi credentials, vCard, GS1 element strings, geo, tel, SMS, email.

13Why it survives bad frames

src/decode.ts · the pass plan

Wrapped around the pipeline is a small amount of strategy. Huge frames get a downscaled pass first. It is cheaper, and the averaging doubles as a low-pass filter that often reads blurry codes better than full resolution does; full resolution still runs when it fails. The inverted image is tried when the normal one yields nothing. tryHarder widens the finder-triple search and adds a 2× pass for oversampled codes. Each trick is a retry, not a redesign. The pipeline itself never changes, which is what keeps decode() under 10 kB brotli (13.4 kB with the whole camera scanner) and half a millisecond on a clean frame.

Numbers to back that up, including head-to-head detection rates against jsQR and @zxing/library, live in the benchmarks section and regenerate from pnpm compare.