Colibri Runs GLM-5.2 Locally on 25GB. Here's the Catch.
A single-file C engine is streaming a 744B-parameter model off an NVMe drive on hardware you already own. I read the repo like an SRE — the receipts are real, and so is the speed tax.
- Colibri runs GLM-5.2 — a 744B-parameter MoE model — on a machine with ~25GB of RAM by streaming its 19,456 routed experts from a ~372GB NVMe container. No GPU, no Python at runtime.
- Author-reported speeds: 0.05-0.1 tok/s cold on the 25GB dev box, ~1.8 tok/s warm on a 128GB CPU desktop, 5.8-6.8 tok/s decode on 6x RTX 5090.
- The MTP speculative-decoding head must be int8 — the original HF mirror ships int4 heads that collapse to 0-4% draft acceptance. Use the mateogrgic mirror.
- Repo went from created (July 1) to 936-point Show HN (July 9) to 17,831 GitHub stars by July 22. Apache 2.0 engine, MIT weights.
Colibri is a pure-C, Apache 2.0 inference engine that runs GLM-5.2, Z.ai's 744-billion-parameter MIT-licensed MoE model, locally on a consumer machine with roughly 25GB of RAM and 400GB of fast NVMe. It keeps the ~17B dense parameters resident at int4 and streams the 19,456 routed experts from disk, with no GPU and no Python at runtime.
Can you run GLM-5.2 locally — all 744 billion parameters — on a machine with 25GB of RAM and no GPU? Colibri says yes, and the receipts in the README hold up. It’s a pure-C engine that keeps the dense ~17B parameters in RAM at int4 (~9.9GB) and streams the rest off your NVMe drive. No Python at runtime. No BLAS. No CUDA.
Let me be straight with y’all: I have not run this on my own metal yet. I don’t have 400GB of NVMe sitting free. This is me reading the repo like an SRE, not faking a benchmark. But I’ve been the loudest GLM 5.2 evangelist on this site since the Fable blackout, when I picked it up through Cline for $2 and watched it do savage pentest work on my own apps. I said then we’d all end up “running back to torrents… on home hardware because the government doesn’t want us to run the latest models.” Colibri is that prediction with a Show HN post.
How does Colibri fit a 744B model in 25GB of RAM?
Sparsity does the work. GLM-5.2 activates only ~40B of its 744B parameters per token, and only ~11GB of that actually changes token-to-token — the routed experts. Colibri keeps the stable dense part resident and streams the 19,456 routed experts (75 MoE layers x 256, plus the MTP head, ~19MB each at int4) from a ~372GB disk container on demand.
That’s the whole trick, and it’s a beautiful one. The router says which experts it needs; the engine pages them in like a database working set. One C file (~6,700 lines as of v1.1.0) plus small headers. The weights themselves are MIT from Z.ai, engine Apache 2.0. Started as a one-person project on a 12-core laptop. Repo created July 1… 936 points on Hacker News July 9… 17,831 stars by July 22. Crazy work.
How fast is GLM-5.2 running locally with Colibri?
Slow at the floor, usable at the ceiling — and the README is refreshingly honest about the ladder. These are the author’s self-reported numbers, not mine and not independent: 0.05-0.1 tok/s cold on his 25GB dev box, up to 5.8-6.8 tok/s decode across six RTX 5090s.
| Hardware | Author-reported speed | Where it loses |
|---|---|---|
| 25GB RAM, 12-core CPU (cold) | 0.05-0.1 tok/s | Interactive anything. That’s ~3,000 tokens in an 8-hour overnight run. |
| 128GB CPU-only desktop (warm) | ~1.8 tok/s | Still no chat pacing — but 50k+ tokens overnight is real batch work. |
| Single RTX 5070 Ti laptop-class box | 1.07 tok/s | GPU barely helps until experts get residency. |
| 6x RTX 5090, full residency | 5.8-6.8 tok/s | If you own six 5090s, you have other options. |
Compare the other local path: Unsloth’s official GLM-5.2 doc puts its 1-bit quant at 223GB of combined RAM+VRAM, 2-bit at 245GB. Different lane entirely — Unsloth wants you rich in memory; Colibri wants you rich in patience.
What’s the catch when you download GLM-5.2 for Colibri?
The MTP head. Colibri’s speculative decoding needs the multi-token-prediction head at int8 — int4 MTP heads collapse to 0-4% draft acceptance, which kills the 2.2-2.8 tokens-per-forward payoff. The README warns the original mirror ships broken int4 heads. Download mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp — ~372GB, keep 400GB free on fast NVMe.
Peep game: that one quantization detail is the difference between speculative decoding paying and doing nothing. Read the README before you burn a weekend of bandwidth.
Should you run GLM-5.2 locally with Colibri?
If you want a frontier-class coder answering in real time, no — my open-weight coding models guide covers API routes that cost pennies. If you want the strongest MIT-licensed model on Earth running on hardware nobody can revoke, throttle, or export-control… this is the door, and it’s Apache 2.0.
I got my start in Ops. Availability you own beats speed you rent.
The hummingbird is slow. It’s also yours.
#TheAIMogul
Bottom lineColibri is the most important local-AI repo of the summer: it proves the strongest MIT-licensed open-weight model can run on hardware you already own, if you'll trade speed for sovereignty. Download it for overnight batch work and the principle — not for interactive chat.