Smaller models.
More possibilities.

SciFact test split, nDCG@10 against the model's own fp32 index (0.7559): 120 MiB at 2.1 bits per weight 98.2 %, 105 MiB at 1.8 bits 96.5 %, 91 MiB at 1.6 bits 93.6 % (torch simulation of the container, synthetic-query calibration, 4-d codebooks per 256-column block, input-side structured rotation). The released SciDocs containers of the same recipe (127 / 113 MiB) hold 94.7 / 91.3 % and were verified in the browser to ±0.001. Below ~180 MiB llama.cpp has no format at all; BitNet-270m gets to 140 MiB only by training the model ternary.

bge-m3 Q4_K_M with a 4-bit token table: cosine to the fp32 query vector 0.990, top-10 overlap 0.864; the 321 MiB Q3_K file keeps 98.3 %. Qwen3-Embedding-0.6B: 98.7 % at 340 MiB, 99.5 % at 385 MiB. The same Czech-calibrated files keep 99.6–100 % on SciFact. The Czech evaluation uses synthetic queries; its full-precision level is the same for every base model we tried, so it measures the compression, not the base.

harrier-0.6b at 2.10 bits per weight in our WebGPU runtime: p50 103 ms, p95 113 ms, load 2.4 s, 77 % of the peak process memory of the llama.cpp path, at −0.011 nDCG@10 against the 3.4-bit file (97.6 % of fp32 on SciFact). One laptop, one integrated GPU, 300 queries; no discrete-GPU or mobile numbers yet. The runtime is not public; the containers are.

SciFact 98.9 %, NFCorpus 99.3 %, ArguAna 101 %, SciDocs 99.2 % (microsoft/harrier-oss-v1-0.6b, GPTQ on the llama.cpp Q3_K grid, generic English calibration). At this bit width llama.cpp's own quantiser reaches the same quality with the same text; the value is in the verification, and we say so. The file stays in the table below because a verified client is useful even when it is not novel.

The idea: give the neurons the corpus uses most a larger codebook and the rest a smaller one at the same average rate. Three rotation seeds, matched random control: importance minus random −0.0085 cosine to fp32, nDCG@10 within noise; using the importance only to weight the codebook fit: −0.008. Side finding: two quantisations that differ only in which rows share a codebook differ by up to 0.03 cosine, so on this grid an effect under 0.01 needs three seeds or a paired construction. Pre-registered and closed 2026-09-10.

The prompt in front of every query is the same 19 tokens, so the full-precision model computes its keys and values once (2 MB) and the compressed model only reads the user's words. The naive version, swapping the exact prompt into an existing client, looked huge (+0.037 at 1.6 bits) but only repaired files calibrated on documents; a client calibrated on synthetic queries reproduces the prompt itself and gains nothing (−0.003 / +0.000 on the released files). Calibrating every block with the exact prompt in place and shipping it is the real thing: +0.013 / +0.009 / +0.008 nDCG@10 over the best query-calibrated client of the same seed at 1.8 bits on SciFact, every paired interval above zero, cosine to fp32 0.90 → 0.92. On SciDocs (short title-like queries) +0.002 / +0.004 / +0.000, on NFCorpus −0.000 / +0.002, on ArguAna +0.001 / −0.009, at 2.1 bits +0.002, at 1.6 bits +0.001: nothing. The cosine to fp32 rises everywhere; the ranking follows only on SciFact, and per query it is a redistribution (rank-1 queries lose, rank-2–3 and outside-top-10 queries gain) that sums to a gain there and to zero elsewhere; we do not know why. Such a client needs the prompt at inference (75–84 % without it). Pre-registered, 2026-09-10/11; the recipe is in the runtime as an option, the published containers are unchanged.