Samsung LittleBit: 13B in 0.84 GB
But Not for Commercial Use
Weights are never stored. Only binarised low-rank factors and three learned scales are. Llama2-13B shrinks 31-fold, a 70B model nearly 70-fold, and the KV cache about 21-fold.Oct 18, 2026
Samsung Labs has published LittleBit, a quantisation method that compresses LLMs into the extreme 0.1 bits-per-weight regime. Llama2-13B drops from 26.06 GB in FP16 to 0.84 GB — about 31× — and even a 70B-class model fits in 1.98 GB. It is not a free lunch, though: there is a quality cliff and a non-commercial licence.
- Verified Confirmed in an official blog, repository or primary document
- Vendor-reported The developer's or vendor's own claim, not independently verified
- Third-party tested Independently checked by an outside developer, outlet or measurer
- AIfrontier view Our own interpretation
1. What got compressedVerified
LittleBit is a NeurIPS 2025 paper (arXiv:2506.13771); its successor LittleBit-2 is an ICML 2026 paper. The implementation is public in SamsungLabs' GitHub repository. Source: GitHub (SamsungLabs/LittleBit) Source: Samsung Research
| Model | FP16 | 0.3 BPW | 0.1 BPW | Reduction |
|---|---|---|---|---|
| Llama2-7B | 13.49 GB | 0.79 GB | 0.63 GB | >21× |
| Llama2-13B | 26.06 GB | — | 0.84 GB | 31.02× |
| Llama2-70B | 138.04 GB | — | 1.98 GB | 69.72× |
"13B under 1 GB" is accurate. The 70B result may matter more in practice.
2. How it works: signs and scales, never the weightsVerified
Ordinary quantisation stores each weight at lower precision — 4, 3 or 2 bits. LittleBit does not store the weight matrix at all.
- Latent factorization. Each dense matrix W is approximated as the product of two low-rank factors (W ≈ UVT), and those factors are binarised to ±1.
- Three scales compensate. What binarisation destroys is restored by learnable FP16 scales: row scale h, column scale g, and the latent scale l, which learns the relative importance of each rank dimension — the paper's key idea.
- Dual-SVID initialisation. Take signs from a truncated SVD, then estimate scales from rank-1 approximations of the factor magnitudes. This gives quantisation-aware training a stable start.
- Residual compensation. Without increasing the bit budget, one higher-rank approximation is reallocated into two lower-rank paths; the residual path captures what the primary path missed, and the two outputs are summed.
Computation becomes two smaller binary matrix multiplications plus element-wise scaling, and the effective full-size weight is never stored. At inference the model architecture is unchanged.
3. Quality: stable until the cliffVendor-reported
WikiText-2 perplexity (lower is better), against the leading prior sub-1-bit method STBLLM.
| Setting | LittleBit | STBLLM |
|---|---|---|
| Llama2-7B @ 0.55 BPW | 10.47 | 30.67 |
| Llama2-7B @ 0.3 BPW | 12.00 | 1.8e3 (collapse) |
| Llama2-7B @ 0.1 BPW | 15.92 | — |
Where STBLLM collapses at 0.3 BPW, LittleBit holds 12.00. At 0.1 BPW it still scores 15.92 — better than STBLLM managed at 0.7 BPW (19.17). Zero-shot common-sense accuracy holds too: 47.26% average for Llama2-7B at 0.55 BPW (STBLLM 44.29%), and 45.20% at 0.3 BPW. Source: Samsung Research
4. Speed and the KV cacheVendor-reported
- Kernel level: 11.6× over FP16 for a Llama2-70B MLP layer at 0.1 BPW (A100, batch size 1).
- End to end: 203.20 tokens/second for Llama2-7B generating 128 tokens, versus 82.56 in FP16 — 2.46×, accelerating linear layers only.
- KV cache: because the key/value projection matrices are factorized, only the latent dimension r needs caching instead of the full model dimension d — roughly a 21.3× reduction for Llama2-7B at 0.1 BPW.
The KV saving arrives as a side effect, and it may be the more useful one: in long-context inference the cache often binds before the weights do.
5. Three constraints before you adopt itThird-party tested
Quality degrades as bit-width falls, with a cliff in the 0.3-0.1 BPW band. 0.1 BPW is demonstrated-to-work territory, not everyday territory. Start evaluation at 0.3-0.5 BPW.
A residual-path ablation on OPT-1.3B improves perplexity from 1.0 down to 0.3 BPW, but inverts at 0.1 BPW — 60.011 with the residual path versus 48.512 without. At the extreme edge, the design's own premise breaks.
The repository is licensed CC BY-NC 4.0. It is open source, but not usable inside commercial products. The method is quantisation-aware training, so you need GPU time to produce a compressed model, not just a download. And lm_head plus the embedding layer stay in FP16, uncompressed.
Supported models: OPT, Llama and Llama 2/3, Phi-4, Qwen2.5 and QwQ, Gemma 2 and 3, Qwen3. Source: alphaXiv (paper walkthrough)
6. What it adds up toAIfrontier view
"13B under 1 GB" is true, but it is a storage claim. Compute does not get 31× faster; measured end-to-end speedup is 2.46× with linear layers accelerated only. Conflating the two leads to disappointment.
The more consequential number may be the 21× KV-cache reduction: in long-context inference the cache often binds first, so 21× headroom there buys a lot of usable context in the same VRAM.
Decide in three steps. (1) Commercial use is not permitted (CC BY-NC 4.0), so scope it to internal research. (2) Do not aim at 0.1 BPW; start at 0.3-0.5. (3) Budget the QAT training cost. Quantisation research has always been a fight against the compression-versus-quality curve; LittleBit pushed that curve to the right. Samsung has since published LittleBit-2 on top of it.
Sources
- GitHub, "SamsungLabs/LittleBit" (official implementation of the NeurIPS 2025 and ICML 2026 papers; CC BY-NC 4.0) https://github.com/SamsungLabs/LittleBit
- Samsung Research blog, "LittleBit: Ultra Low-Bit Quantization via Latent Factorization" https://research.samsung.com/blog/LittleBit-Ultra-Low-Bit-Quantization-via-Latent-Factorization
- arXiv, "LittleBit: Ultra Low-Bit Quantization via Latent Factorization" (2506.13771) https://arxiv.org/abs/2506.13771
- alphaXiv paper walkthrough (quality cliff, uncompressed lm_head, residual reversal, kernel vs end-to-end speed) https://www.alphaxiv.org/abs/2506.13771
- OpenReview, "LittleBit-2: Maximizing the Spectral Energy Gain in Sub-1-Bit LLMs via Latent Geometry Alignment" (ICML 2026) https://openreview.net/pdf?id=9TMdlXCn79
- This site, "Local and Cloud LLMs" (already references LittleBit-2 and Joint-ITQ)