How to Self-Host Flux 2 in 2026: Real VRAM Numbers for Every GGUF Tier, and Whether to Upgrade From Flux.1-dev

By RunAIHome Team · 12 min read
fluxcomfyuiimage-generationgpuvramgguftutorial

TL;DR: Flux 2 is really three models: the 32B FLUX.2 [dev] (needs 19-20GB just for a Q4 GGUF), and the Klein 4B/9B distills that actually fit consumer cards. On 24GB, run dev at Q4_K_M; on 16GB or less, run Klein and stop feeling bad about it — the 4B is the only Apache 2.0 weight in the family anyway.

What you’ll be able to do by the end:

  • Pick the right Flux 2 checkpoint and quant for your exact VRAM (8, 12, 16, 24, or 32GB), from verified file sizes — not vibes
  • Set up the ComfyUI workflow: transformer, Mistral text encoder, VAE, correct folders, and the update step everyone skips
  • Decide with actual benchmark data whether Flux 2 output justifies replacing a working Flux.1-dev pipeline

Honest take: If you have 24GB and image quality pays your bills, the upgrade is real — Flux 2 dev wins 66.6% of head-to-head text-to-image comparisons and finally renders legible text. If you’re on 16GB or less and Flux.1-dev works for you, Klein 4B is the only variant worth the download, and even that’s optional.

Black Forest Labs shipped FLUX.2 [dev] on November 25, 2025 — a 32B-parameter text-to-image and editing model, more than 2.5× the size of Flux.1-dev’s 12B. The [klein] family followed on January 15, 2026: a 4B under Apache 2.0 and a 9B under the FLUX non-commercial license, both distilled from dev. Eight months on, the quantization ecosystem has settled — Unsloth and city96 GGUFs cover every tier from 12.9GB to 35GB, ComfyUI runs everything natively, and there’s enough community benchmark data to answer the question that actually matters: what fits on your card, and is it better than what you’re already running?

This is the guide we wished existed when Flux.1 first landed: file sizes from the actual repos, generation times from real hardware, and the license fine print up front.

The Flux 2 family, sorted by what you can legally do with it

ModelParamsLicenseReleasedRealistic minimum VRAM
FLUX.2 [dev]32BFLUX.2-dev Non-CommercialNov 25, 202516GB (Q3 GGUF, tight)
FLUX.2 [klein] 9B9BFLUX Non-CommercialJan 15, 202616GB (FP8)
FLUX.2 [klein] 4B4BApache 2.0Jan 15, 20268GB

The license column deserves more attention than it usually gets. Only the Klein 4B is Apache 2.0 — output you can sell, weights you can fine-tune commercially, no strings. Both the 32B dev and the 9B Klein are non-commercial: fine for a home lab, not fine for client work without a paid BFL license. We covered why this trips people up in our local image-gen licensing breakdown — the short version is that the best-looking checkpoint in a family is almost never the commercially usable one.

One architectural change matters for your hardware planning: Flux 2 replaces Flux.1’s T5-XXL/CLIP text encoders with Mistral 3 Small. That’s a full LLM riding shotgun, and it’s why the system RAM floor moved: 32GB minimum, 64GB recommended, with roughly 60GB of disk for a full dev setup (transformer + encoder + VAE).

FLUX.2 dev: the real VRAM table

The 32B dev transformer weighs about 64GB in BF16 — H100 territory, not consumer hardware. The Comfy-Org FP8 build (made with NVIDIA) is about 35GB as a file, which still doesn’t fit any consumer card whole. What makes dev runnable at home is GGUF. Verified file sizes from the Unsloth FLUX.2-dev-GGUF repo, with city96’s Q4_K_S alongside:

QuantFile sizeFits fully on
Q8_035 GBNothing consumer (offload required)
Q6_K27.4 GBRTX 5090 32GB
Q5_K_M23.9 GBRTX 5090 (4090 with offload)
Q4_K_M20 GBRTX 4090 / 3090 24GB
Q4_K_S19 GBRTX 4090 / 3090 24GB
Q3_K_M15.8 GB16GB cards, tight
Q2_K12.9 GB16GB cards comfortably; quality suffers

Mapping that to actual cards:

24GB — RTX 4090 and RTX 3090. Q4_K_M (20GB) or Q4_K_S (19GB) is the sweet spot: transformer fully resident, text encoder offloaded to system RAM. Community benchmarks on an RTX 4090 running the Q8_0 GGUF with partial offload clock 15-17 seconds per 1024×1024 image at 20 Euler steps — Q4 runs faster since more of the model stays on-die. That’s your reference number for “is my setup healthy.”

32GB — RTX 5090. Q5_K_M (23.9GB) or Q6_K (27.4GB) resident with room for the VAE. Blackwell cards also get the NVFP4 path — the FLUX.2-dev-NVFP4 build is a ~14GB footprint with hardware-accelerated 4-bit math, and it’s the fastest way to run dev on a 5090. We benchmarked that pipeline separately in the NVFP4 ComfyUI speed guide.

16GB — RTX 5060 Ti 16GB, 5080, 4080. Q3_K_M at 15.8GB technically loads, but you’re one context spike from an OOM. Q2_K (12.9GB) fits with headroom and visibly degrades fine detail — at which point Klein 9B at FP8 gives you better output from a model that was actually designed for this tier. That’s the honest recommendation.

12GB and 8GB. Dev is off the table. Klein territory — next section.

Klein: what 8-16GB cards should actually run

The Klein models aren’t crippled afterthoughts; they’re distills BFL built specifically for consumer hardware, and the speed numbers are the appeal:

  • Klein 4B: ~13GB file in BF16, ~7GB in FP8. Per BFL’s own repo, it fits in ~8GB VRAM, and it generates a 1024×1024 image in about 1 second on an RTX 4090. Sub-second to one-second generation changes how you use the tool — iteration becomes interactive instead of batch-and-wait.
  • Klein 9B: needs ~29GB of working VRAM at full FP16 (that’s 5090/Mac territory), but FP8 brings it to 16GB cards. On an RTX 4090 at FP16 it generates in ~3.7 seconds at 28 steps — the quality-to-latency sweet spot of the family.

On a 12GB card like the RTX 3060, Klein 4B BF16 is tight but workable with the encoder offloaded; FP8 is comfortable. On 8GB, use FP8 or the GGUF builds and expect encoder offload to add a couple of seconds per prompt on the CPU side.

If your card predates this tier entirely, Flux.1-dev still runs on 8GB via GGUF Q4 at ~6.8GB — Flux 2 didn’t retroactively make it worse.

ComfyUI setup: files, folders, and the update everyone skips

ComfyUI supports Flux 2 natively — no custom nodes for the standard workflow (GGUF loading still uses the ComfyUI-GGUF extension). Three files, three folders:

ComfyUI/models/
├── diffusion_models/
│   └── flux2-dev-Q4_K_M.gguf              # or your chosen quant
├── text_encoders/
│   └── mistral_3_small_flux2_fp8.safetensors
└── vae/
    └── flux2_vae.safetensors

Grab the transformer quant from Unsloth or city96, and the text encoder + VAE from the Comfy-Org flux2-dev repo (split_files/ directory). Take the FP8 text encoder, not BF16 — it’s a 24B-class Mistral encoder, and the BF16 version alone can eat more memory than your transformer.

huggingface-cli download unsloth/FLUX.2-dev-GGUF \
  flux2-dev-Q4_K_M.gguf \
  --local-dir ComfyUI/models/diffusion_models/

Then update ComfyUI before you load anything. Flux 2 node support shipped in late-November 2025 builds, and the GGUF loader gained FLUX.2 architecture support in ComfyUI-GGUF updates after that (the feature request thread is where to check current status). A stale install fails with model-detection errors that look like corrupt downloads but aren’t.

On a healthy launch you’ll see the model land where it should:

$ python main.py
Total VRAM 24564 MB, total RAM 65536 MB
Using pytorch attention
loaded completely ... flux2-dev-Q4_K_M.gguf
Requested to load MistralTEModel
loading in lowvram mode

That lowvram mode line on the text encoder is correct behavior, not a warning — the Mistral encoder runs once per prompt, so parking it in system RAM costs you a second or two per generation and saves ~12GB of VRAM.

The OOM you’ll actually hit, and the fix

The most common failure on 24GB cards looks like this mid-generation:

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 2.50 GiB.

Three fixes, in order of how often they work: (1) you downloaded the BF16 text encoder — swap it for mistral_3_small_flux2_fp8.safetensors; (2) launch with --lowvram so ComfyUI streams weights instead of front-loading them; (3) drop one GGUF tier (Q4_K_M → Q4_K_S buys you a gigabyte). And if you’re on Windows and the process dies with OS error 1455 instead of a CUDA error, that’s your paging file, not your GPU — we have a dedicated fix for that one.

Is it actually better than Flux.1-dev?

The upgrade question deserves numbers, not marketing. What the data says:

  • In fal.ai’s head-to-head win-rate testing, FLUX.2 dev wins 66.6% of text-to-image comparisons versus 48.1% for comparable models — a consistent, measurable gain over Flux.1 and contemporary open-weight systems, not a coin flip.
  • Text rendering is the headline fix. Flux.1’s misspelled-poster problem is mostly gone: Flux 2 produces legible, consistent text on posters, packaging, and UI mockups. If your use case involves any in-image text, this alone justifies the switch.
  • On quality-vs-cost ELO charts, FLUX.2 dev clusters in the 1030-1050 band — near-frontier quality at open-weight prices.
  • Editing is built in. Flux 2 dev handles multi-reference editing natively, where Flux.1 needed the separate Kontext model for image-conditioned edits.

Now the other side of the ledger. Flux.1-dev runs in ~12GB at FP8; the equivalent-quality Flux 2 dev quant needs 19-20GB plus a heavier encoder and twice the system RAM. You’re paying a full GPU tier for the quality jump. On a 24GB card that’s fine. On 16GB, “upgrading” means either a quant tier that erases the quality gains (Q2_K) or switching to Klein 9B — which is a sidegrade with better text rendering, not a straight upgrade.

So: 24GB+ and quality-sensitive work → upgrade, keep Q4_K_M as your daily driver. 16GB or less with a working Flux.1 pipeline → try Klein 4B for the speed, keep Flux.1-dev for finals, skip dev entirely.

No GPU, or want to test before committing 60GB of downloads?

A RunPod RTX 4090 instance runs about $0.34-0.69/hour depending on tier — two dollars of testing tells you whether Flux 2 dev output justifies rearranging your hardware budget, and RunPod publishes an official FLUX.2 deployment guide with a ready ComfyUI template. If you find yourself renting more than ~10 hours a month, the rent-vs-buy math starts favoring a used 3090.

FAQ

Can I use Flux 2 dev output commercially? Not under the default license — FLUX.2 [dev] is non-commercial, same as the 9B Klein. The Klein 4B is Apache 2.0, and its output is yours to sell. BFL sells commercial licenses for the dev weights.

Does Flux 2 work on a used RTX 3090? Yes, and it’s arguably the value play: 24GB fits Q4_K_M (20GB) the same as a 4090, just slower. Ampere doesn’t get FP8 or NVFP4 acceleration, so GGUF is the right format for a 3090.

How much system RAM do I really need? 32GB minimum, 64GB recommended. The Mistral text encoder offloads to system RAM in every consumer-VRAM workflow, and Q4-class transformer + FP8 encoder + OS overhead lands right around the 32GB line.

Do I need custom nodes in ComfyUI? Native safetensors workflows: no, just a current ComfyUI build. GGUF quants: yes, the ComfyUI-GGUF custom node, updated to a version with FLUX.2 support.

Is Klein 4B better than Flux.1-dev? Different trade. Flux.1-dev (12B) still edges out Klein 4B on raw detail; Klein generates in ~1 second on a 4090, renders text far better, and is Apache 2.0 (Flux.1-dev is non-commercial). For commercial output or rapid iteration, Klein wins. For maximum-quality personal work on 12-16GB, Flux.1-dev remains competitive.

What about Apple Silicon? The GGUF quants load through ComfyUI on M-series unified memory — a 48GB+ M4 Max fits Q8_0 outright. Generation is materially slower than a 4090; Klein 9B is the more sensible Mac daily driver.

Sources

Last updated August 9, 2026. Prices, quant availability, and ComfyUI support status change quickly; verify current requirements before downloading 60GB of weights.

  • RTX 4090 — the 24GB workhorse for FLUX.2 dev Q4_K_M
  • RTX 5090 — 32GB for Q5/Q6 quants and the NVFP4 fast path

Was this article helpful?