FlowBlock

Wavefront-Parallel Decoding for Self-Correcting Diffusion Language Models

FlowBlock is a training-free parallel decoding framework for block-wise diffusion language models (dLLMs). Block-wise dLLMs such as LLaDA-2.x decode one block at a time, which makes inter-block decoding strictly serial. FlowBlock observes that the token-to-token (T2T) self-correction ability of LLaDA-2.1 series enable a downstream block start from an informative draft instead of a finalized predecessor — so block finality becomes a scheduling resource rather than a hard dependency.

FlowBlock is built on two mechanisms:

• Gated Wavefront Decoding (GWD) — admits blocks into a bounded wavefront only when a readiness gate (SPAWN_TH) is satisfied, jointly refines active blocks via T2T editing, and commits blocks in order under a windowed block-causal mask that preserves exact frozen-prefix KV-cache reuse.
• Heterogeneous Wavefront Packing (HWP) — gives each request its own wavefront while packing all active windows into dense, shape-stable batched forwards, so asynchronous progress stays GPU-efficient in batched serving.

✨ Highlights

• Up to 2.95× / 4.01× higher TPS than LLaDA-2.1 / LLaDA-2.0 under batched serving (1.57× / 3.17× at batch size 1), with up to 53.6% / 77.1% lower latency.
• Accuracy preserved or improved: +1.3 points on average across eight math / code benchmarks.
• No training: runs the unmodified LLaDA-2.1 series weights.

🛠️ Installation

Requirements: Linux, NVIDIA GPU(s) with ~80 GB memory for the 16B-A1B MoE checkpoints, CUDA 12.x driver.

git clone  FlowBlock
cd FlowBlock

conda create -n flowblock python=3.11 -y
conda activate flowblock

# 1) Install PyTorch matching your CUDA version (validated: 2.8.0 + cu128)
pip install torch==2.8.0 --index-url https://download.pytorch.org/whl/cu128

# 2) Install the base dependencies (sglang, lm-eval, ...)
pip install -r requirements.txt

# 3) Install vLLM last — this is a separate step on purpose
pip install vllm==0.10.2

The dinfer package is used directly from source — the evaluation scripts set PYTHONPATH automatically, no pip install step is needed. Verify the setup:

PYTHONPATH="$PWD/dInfer/python" python -c \
  "from dinfer.decoding import FlowBlockJointDiffusionLLM, flowblock_packed; print('Flo