TL;DR: Cactus Compute’s Needle 2 is a 45-million-parameter tool-calling model compressed to a single 14MB binary that runs a full session in about 28MB of RAM — vendor-reported decode is 500 tokens/sec on a Raspberry Pi 5. It does tool calling, device control, and structured extraction, not conversation. That makes it the cheapest credible front-end tier for a home lab where the real model lives on your GPU server.

	Pi 5 2GB + Needle 2	Pi 5 4GB + LFM2.5-230M	Used RTX 3090 server
Hardware cost (Aug 2026)	~$50	~$110	$700+ system
Idle power	~3W	~3W	20–30W idle, 100W+ under load
Electricity, 24/7 at $0.12/kWh	~$3/yr	~$3/yr	$20–$100+/yr
What it runs	Tool calls, extraction at 500 tok/s	Extraction, routing, short summaries at 42 tok/s	27B–35B chat/coding models
The catch	256-token sliding window; no chat, no reasoning	Needs ~1GB RAM; 12× more FLOPs/token	Price, noise, power bill

Honest take: Don’t install Needle 2 expecting a small assistant — install it if you want a $50 board that turns “make it 21 and cool the room” into a validated function call in a fraction of a second, and escalates anything it isn’t sure about to the big model on your GPU rig. The confidence gate is the feature; the 14MB is just how it gets there.

On August 13, 2026, cactus-compute/needle hit GitHub’s trending page with a claim that reads like a typo: a foundation model in 14 megabytes. Three days later the repo sits at 6.2k stars, and the claim holds up — with an asterisk the size of the model itself. Needle 2 is real, it’s MIT-licensed, and it genuinely runs agentic tool-calling loops on hardware that costs less than a mid-range mechanical keyboard. What it is not is a general-purpose LLM, and understanding exactly where that line sits is the difference between a genuinely useful edge node and a toy you uninstall in a week.

We’ve covered the tiny end of local AI twice before — the 28.9M-parameter model on an $8 ESP32 and LFM2.5-230M on a Raspberry Pi 5 — and Needle 2 slots into a different niche than either. Here’s the full picture.

What Needle 2 actually is

Needle 2 is a 45M-parameter model trained by Cactus Compute for three jobs: tool calling, device use, and structured data extraction. Per the project README, the architecture is a “Simple Attention Network” — a transformer variant with a Hadamard MLP in place of the standard FFN, grouped-query attention, an “engram” key-value memory, and multi-lane hyper-connections. The weights are trained and deployed at what Cactus calls CQ2-bit (“Cactus Quants” 2-bit) — so the 14MB figure is the quantized production artifact, not an FP16 checkpoint. There is no separate GGUF to download and no runtime to install: the weights are baked into the company’s own C++ engine, and the whole thing ships as one binary that runs a session in roughly 28MB of RAM.

The lineage matters for setting expectations. The first Needle, released earlier in 2026, was a 26M-parameter model distilled from Gemini 3.1 Flash Lite’s tool-calling behavior — pretrained on 200 billion tokens over 27 hours on 16 TPU v6e chips, then post-trained on a 2-billion-token synthetic function-call dataset in 45 minutes. That’s the entire training run. Needle 2 grows the parameter count to 45M and widens the scope from single-shot function calls to multi-step agentic loops, but the DNA is the same: this is a distilled specialist, not a shrunken generalist.

If you saw the queue-level description “14MB foundation model” and assumed 14MB was a marketing number for a much larger download — reasonable, given how often that happens — it isn’t. The binary is 14MB. The Hugging Face card and the repo agree.

The speed numbers, and what to trust

Cactus reports the following decode throughput for Needle 2:

Device	Decode speed (vendor-reported)
Raspberry Pi 5	~500 tok/s
Sub-$200 phones (Samsung A-series class)	300–700 tok/s
Meta Quest 3S / Apple Vision Pro	400–1,500 tok/s

Flag on the play: these are vendor numbers, published by Cactus on its site and model card, and we haven’t seen an independent Pi 5 replication yet. That said, the claim is physically plausible rather than magical. Decode speed on CPU is memory-bandwidth-bound — you read the active weights once per token — and at 2-bit quantization the entire 45M-parameter model is 14MB of reads per token, versus hundreds of megabytes for a 1B-class model at Q4. Cactus quotes roughly 70 MFLOPs per token for Needle 2 against about 460 for LFM2.5-230M, a 6.5× compute gap that lines up with the size ratio. For calibration from our own coverage: LFM2.5-230M decodes at 42 tok/s on a Pi 5, TinyLlama 1.1B manages ~14 tok/s, and 3B models drop to 4–7 tok/s. A model 16× smaller than LFM2.5 running ~12× faster is the expected shape of that curve.

The practical takeaway isn’t the absolute number anyway. At 500 tok/s, a 30-token function call materializes in ~60 milliseconds. At even a fifth of that speed it would still feel instant. For the workloads Needle 2 targets, latency stopped being the constraint — accuracy is.

The benchmarks: honest wins, honest losses

Cactus’s headline is that Needle 2 “trades wins” with FunctionGemma 270M, LFM2.5-230M, and Apple’s Foundation Model while being 5–70× smaller and running 2-bit against their FP16. The published numbers back the phrase “trades wins” — including the losses:

Benchmark	Needle 2 (45M)	FunctionGemma (270M)	LFM2.5 (230M)
Seal-Tools in-domain	32.6%	16.3%	26.9%
Mobile Actions	63.7%	64.0%	69.1%
BFCL v4 overall	42.6%	46.1%	60.8%
Compute per token	~70 MFLOPs	—	~460 MFLOPs

Read the last two rows before the first. On BFCL v4 — the broadest function-calling benchmark of the three — LFM2.5-230M beats Needle 2 by 18 points. That’s not a rounding error; a 230M model with FP16 weights simply generalizes better across diverse, multi-turn scenarios. Needle 2’s wins come where the task is narrow and the tool schemas are explicit: it leads both Seal-Tools splits outright and posts 98.3% function-name accuracy on Mobile Actions (picking the right tool, as distinct from filling every argument perfectly).

So the honest framing: Needle 2 is the best model per megabyte ever shipped for this task class, and it is not the best small model. If your device has a spare gigabyte of RAM, LFM2.5-230M remains the stronger tool caller. If your budget is 28MB — or you want the model resident 24/7 without evicting anything else — Needle 2 has no competition at all.

Running it on a Raspberry Pi 5

This is the part that surprised us most: there’s no cross-compilation, no build-from-source, no quantization step. Cactus ships manylinux aarch64 wheels, so on a Pi 5 running 64-bit Raspberry Pi OS it’s:

pip install cactus-needle

A minimal thermostat agent, straight from the API docs:

import needle

@needle.tool
def set_thermostat(temperature: float, mode: str):
    "Set the thermostat temperature and mode."
    return {"ok": True, "temperature": temperature, "mode": mode}

agent = needle.Needle(tools=[set_thermostat])
result = agent.run("make it 21 and cool the room")

The response object reports its own telemetry — on desktop-class hardware the docs show output like:

"prefill_tps": 4300.0,
"decode_tps": 850.0,
"peak_ram_mb": 28.5,
"confidence": 0.94

That confidence field is the design decision that makes Needle 2 interesting for home labs rather than just impressive. Every call is confidence-gated: the model emits a calibrated score, and a call is accepted only when it clears your threshold — in Cactus’s words, “the failure mode is escalation, not wrong execution.” Argument schemas are enforced too, with typed constraints (needle.Field(gt=0, le=10000), regex patterns on strings), so a hallucinated $999999 transfer never leaves the sandbox even before the confidence check.

The first trap you’ll hit, and its fix: declare more than five tools and your agent seems to “forget” some of them — calls that worked in isolation stop being made. That’s not a bug; it’s the documented tool-retrieval behavior. Above five declared tools, Needle 2 ranks them and only the five highest-scoring tools enter the context for a given turn. The fix is to write genuinely descriptive tool docstrings (retrieval ranks against them) and to group related functions into one tool with a mode argument instead of five near-duplicates. The same budget-consciousness shows up in the context handling: a 256-token sliding window with tool definitions pinned as KV sinks, and defaults of max_steps=8 and max_new_tokens=256 on the agentic loop. Long documents don’t fit; that’s what the escalation path is for.

Needle 2 vs the ESP32 model vs LFM2.5: three different jobs

The $8 ESP32 project we covered in July runs a 28.9M-parameter model — 64% the size of Needle 2 — but the comparison ends at the parameter count. That model writes TinyStories-grade prose on a microcontroller with 512KB of SRAM; it can’t call a function, answer a question, or follow an instruction. Needle 2 can’t run on that hardware at all: it needs a real OS and a Python runtime, so the floor is Pi-class ARM boards (Pi 4 and 5 included via the aarch64 wheels), phones, or anything x86. If the queue in your head has “Arduino/ESP32” on it, cross that off — 28MB of session RAM is tiny by PC standards and impossible on a $8 microcontroller with 8MB of PSRAM.

Against LFM2.5-230M on the same Pi, the trade is capability versus footprint. LFM2.5 handles short summaries, freer-form extraction, and multi-turn tool use measurably better (that 18-point BFCL gap), needs under 1GB of RAM, and runs 42 tok/s — comfortably past the ~7–10 tok/s human reading speed. Needle 2 gives up the general ability, keeps the structured-task accuracy, and shrinks the footprint 30×. On a Pi that does nothing else, run LFM2.5. On a Pi that’s also your Home Assistant box, your Pi-hole, and your Zigbee bridge — where a resident gigabyte is a real cost — Needle 2 is the one that coexists.

The two-tier home lab: this is the actual use case

Here’s where Needle 2 earns a place in a home lab rather than a bookmark. The pattern we sketched in the ESP32 article — a tiny always-on tier that handles the cheap decisions locally and forwards the hard ones — was hypothetical then because a TinyStories model can’t make decisions. A confidence-gated tool caller can. The architecture:

• Tier 1 (always on, ~3W): A Raspberry Pi 5 — even the $45 1GB board clears 28MB with room to spare — runs Needle 2 as the front door. Smart-home commands, notification triage, receipt/data extraction, wake-word-to-action: handled on the board in tens of milliseconds, no network round trip.
• Tier 2 (on demand, 350W): When the confidence score comes back below threshold — ambiguous phrasing, a request outside the tool schemas, anything that needs actual reasoning — the query escalates over your LAN to the RTX 3090 box running a 27B–35B model (see what 24GB actually runs).

The economics are what make this more than an aesthetic preference. The Pi tier costs about $3 a year in electricity at $0.12/kWh; a 3090 rig that stays awake to catch thermostat commands burns an order of magnitude more doing mostly nothing. Under 2026’s memory-crisis pricing — the Pi 5 4GB is now ~$110 and the 8GB ~$175 after two price hikes, up 83% and 119% from launch MSRP per Tom’s Hardware — a model that’s happy on the $45–$50 low-memory boards is quietly the only tier of Pi whose value proposition improved this year. The GPU tier you probably already own; if you’re speccing it from scratch, start with the GPU buying guide.

Cactus clearly built for this split: escalation-on-low-confidence is the documented failure mode, not a pattern you bolt on. Your escalation handler is just Python — catch the below-threshold result, POST the raw utterance to the Ollama or llama.cpp endpoint on your GPU box, done. For the FOSS tooling on the heavy tier — inference servers, routers, open datasets for fine-tuning your own tier-1 specialist — aifoss.dev covers that side well. And if you want the model to know your tools rather than generic ones, the repo ships a finetuning guide — at 45M parameters, local fine-tuning is a realistic evening project rather than a cloud bill.

What Needle 2 cannot do

Being clear-eyed, because the 6.2k-star hype cycle won’t be:

• No conversation. It calls tools, extracts fields, and controls devices. Ask it a question and you’re outside the training distribution.
• No reasoning. Anything requiring multi-hop thought escalates or fails. The confidence gate makes it fail loudly, which is the best you can ask of a 45M model.
• 256 tokens of sliding context. Long emails, documents, or transcripts don’t fit. Extraction works on receipts, not contracts.
• Five tools per turn. Bigger tool libraries get retrieval-filtered; design around it.
• 2-bit weights are not free. The BFCL v4 gap against FP16 rivals is real. Where accuracy matters more than footprint, spend the RAM.
• Ecosystem lock-in, mild but present. No GGUF, no Ollama tag, no llama.cpp path — the model exists only inside Cactus’s engine via pip install cactus-needle. MIT license softens this, but you’re on their runtime.

FAQ

Is the 14MB figure the real download size? Yes. It’s a single binary with CQ2 2-bit weights baked in — model and engine together. A full session runs in ~28MB of RAM.

Will it run on a Raspberry Pi 4, or only the Pi 5? The wheels are generic aarch64 Linux, so Pi 4 works — expect proportionally lower throughput (the Pi 4’s memory bandwidth is roughly half the Pi 5’s). ESP32/Arduino-class microcontrollers are out: Needle needs an OS and Python.

Is it actually better than FunctionGemma 270M? On Seal-Tools, clearly (32.6% vs 16.3%). On Mobile Actions they’re within half a point. On BFCL v4, FunctionGemma leads by 3.5 points and LFM2.5-230M leads by 18. Needle 2 wins per-megabyte, not outright.

Can I fine-tune it on my own smart-home tools? Yes — the repo documents local fine-tuning, and at 45M parameters it doesn’t need a data-center GPU. Constrained argument schemas plus a few hundred examples of your real commands go a long way.

Does anything leave my network? No. Inference makes no network calls, and the escalation path is whatever you write — pointing it at your own GPU server keeps the entire stack on your LAN.

Recommended Gear

• Raspberry Pi 5 — the always-on tier-1 board; for Needle 2 alone, the cheap 1–2GB variants are enough
• RTX 3090 — the used 24GB workhorse for the tier-2 escalation server

Sources

• cactus-compute/needle — GitHub
• Needle 2 API documentation — GitHub
• Cactus-Compute/needle2 — Hugging Face
• Needle 2 — The 14MB Agentic LLM for Tiny Devices — Cactus Compute
• Needle: We Distilled Gemini Tool Calling into a 26M Model — Cactus Compute
• Show HN: Needle2 — Hacker News
• Meet Needle 2: An Open 45M-Parameter Tool-Calling Model — MarkTechPost
• Raspberry Pi 5 price increases drastically as AI shortage bites — Tom’s Hardware
• 1GB Raspberry Pi 5 now available at $45, and memory-driven price rises — Raspberry Pi
• Cactus v1: Cross-Platform LLM Inference on Mobile — InfoQ

Last updated August 16, 2026. Prices and specs change; verify current rates before purchasing. Speed figures for Needle 2 are vendor-reported pending independent replication.