NVIDIA's TensorRT LLM VisualGen speeds FLUX 3 Image Klein 3.46x on one B200, cutting 2048x2048 text-to-image from 22.36 to 6.46 seconds
A tutorial measuring each optimization for Black Forest Labs' image model, from NVFP4 to step caching and multi-GPU parallelism, with the quality cost of each.
// Key points
- FLUX 3 Image Klein combines Qwen3-VL-4B-Instruct for text conditioning, a diffusion transformer and an autoencoder, with Base and distilled checkpoints
- NVFP4 alone is up to 1.62x faster for text-to-image and 1.58x for image-to-image than BF16 on a B200
- Combined optimizations reach 3.46x for text-to-image (22.36 to 6.46 s) and 4.45x for image-to-image (54.64 to 12.28 s); Cache-DiT alone reaches 1.94x at 2048x2048 with an LPIPS of 0.033
- On eight B200 GPUs, median text-to-image latency falls from 14.90 to 3.00 seconds for Base and from 2.73 to 1.08 seconds for the distilled model
Builder's takeFor PandaClaws' cover images the order of operations is clear from this: try step caching first, since it nearly halves latency at the smallest quality change, then NVFP4; multi-GPU only pays off for interactive editing. I'd rerun the LPIPS comparison on our own prompts before trusting the 0.033.
// Background · from #Open models
Full timeline →- Nov 7 Foundry Local runs Qwen3.8-27B at 4-bit in about 18 GB, on 24 GB consumer GPUs with up to 64K context, as a GitHub Copilot model provider
- Nov 5 Hugging Face proposes Model Integration Contracts for Transformers: tiny real-architecture fixtures on CPU that would have caught a UMT5 bug that sat in main for almost two months
- Nov 5 IBM Research's Palette builds slide decks with a 20B open-weight model and one LoRA adapter, scoring 59.7 on PresentBench versus 60.8 for NotebookLM