为什么值得关注

它提供了一个端到端、哈希固定的 vLLM 大规模推理服务复现套件，包含精确的 GPU 顺序、风扇策略、基准测试工具和现场证据快照，使配置漂移可见且可恢复。

适合谁使用

在 vLLM 上部署大型 GLM 模型的多 GPU 运维人员
关注可复现推理基准测试的 ML 基础设施工程师
使用 NVIDIA Blackwell GPU 进行 LLM 服务的团队
验证 CUDA graphs 和 MTP 工作负载的 Pop!_OS/Ubuntu 服务器管理员

典型使用场景

在类似硬件上复现已验证的四 GPU GLM-5.2-EXL3 推理服务配置
运行持续 C1/C2/C4 热基准测试，测量解码速度、提示处理速度和 TTFT
验证运行主机是否匹配已知良好的镜像、进程参数、GPU/CPU 策略和风扇设置
通过捕获的快照和 Local Studio recipe 审计配置漂移

项目优势

固定不可变镜像摘要和精确 compose 文件，支持可复现部署
自动化脚本覆盖安装、启动、验证、基准测试和现场状态捕获
快照证据包含容器检查、进程参数、镜像元数据、硬件清单和服务状态
基准测试包含 SSE 空闲和引擎停摆看门狗，可保留日志并重启固定容器

使用前须知

需要恰好四块可见的 NVIDIA GPU 以及指定的模型/缓存目录，虽然路径可覆盖
CPU 风扇控制服务是主板相关的，仅在 ASUS Pro WS WRX90E-SAGE SE 上验证
该套件固定到特定 vLLM 镜像和 2026 年 7 月 30 日验证的配置，迁移需要重新验证

README 快速开始

GLM-5.2 EXL3 four-GPU reproduction bundle

This bundle reproduces the live Pop!_OS configuration validated on 30 July 2026. It runs GLM-5.2-EXL3 on four RTX PRO 6000 Blackwell GPUs, keeps normal high-reasoning behavior, uses MTP3, and preserves the server's full and piecewise CUDA graphs. It does not use eager execution, disable CUDA graphs, add stop sequences, or impose a generation token cap.

Known-good operating point

Server image: voipmonitor/vllm:gilded-gnosis-v20-vllmb46c3aa-si35aebc6-fi801d57a-cu132-20260730-r12@sha256:bcd65c13076b00cf7804001ce70085453de958b082897b9674a26d9fab4d436a
Model: GLM-5.2-EXL3-TR3-3.0bpw
GPUs: four RTX PRO 6000 Blackwell, 250 W limit each
CPU: boost enabled, 3.1 GHz maximum, powersave governor, balance_performance EPP
CPU fan policy: 60% at 20 C, 80% at 40 C, 100% at 55 C and above
GPU fan policy: 50% at 30 C, linear aggressive ramp, 100% at 80 C
Safety stops in benchmark: CPU 88 C, GPU 82 C
Serving: TP4, DCP4, ag_rs, MTP3 greedy
KV cache: NVFP4 DSA MLA KV, configured by the pinned image launcher
Context: 140,000 maximum model length
Scheduler: maximum 8 sequences, 3,072 batched tokens, async off
GPU memory utilization: 0.95
CUDA graphs: graph ceiling 32; effective runtime capture sizes are recorded in snapshot/effective-process-args.txt and the server log
PCIe experimental paths: disabled

The exact host-bound compose file is docker-compose.exact-popos.yml. It preserves the validated GPU UUID order. Use docker-compose.portable.yml for a similar four-GPU machine; it maps the four GPUs as CUDA devices 0 through 3.

Prerequisites

Pop!_OS or Ubuntu with a working NVIDIA driver, Docker Engine, NVIDIA Container Toolkit, Docker Compose v2, curl, Python 3, ipmitool, and systemd.
Exactly four visible NVIDIA GPUs.
The EXL3 model at /mnt/llm_models/GLM-5.2-EXL3-TR3-3.0bpw, or set GLM52_MODEL_DIR.
A writable cache directory at /mnt/llm_models/cache_glm52_exl3_text, or set GLM52_CACHE_DIR.
Python 3 with the venv module. The installer creates an isolated /opt/glm52-host environment and installs the hash-pinned NVML binding.
The raw CPU fan command is board-specific and was validated on the ASUS Pro WS WRX90E-SAGE SE. Do not install that service on another board until its IPMI command and readback format are verified.

On

项目描述

Reproducible GLM-5.2 EXL3 MTP3 serving and thermal profile for 4x RTX PRO 6000 Blackwell

glm52-exl3-4x-rtx-pro-6000