Krishnagangwal
Featured
CS-Fundamentals
A curated collection of Computer Science fundamentals (PDFs, notes, cheatsheets, interview question banks) for placement preparation, covering seven core subjects plus general resources.
Inkling-Small-NVFP4-DSpark-BF16-KV-262K-2x-DGX-Spark
This repository provides reproducible vLLM recipes, scripts, and code mods to serve thinkingmachines/Inkling-Small-NVFP4 with a DSpark draft across two DGX Spark nodes, including verified benchmark results and documented failure modes.
Stars
4
7-day growth
No data
Forks
0
Open issues
0
License
No data
Last updated
2026-07-31
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It combines a real two-node DGX Spark deployment with measured speculative-decoding throughput, custom vLLM modifications to support DSpark, and honest reporting of a C6 engine failure.
Who it is for
- Developers deploying Inkling-Small-NVFP4 or similar models on DGX Spark
- vLLM inference engineers working with speculative decoding or DSpark
- MLOps teams building reproducible multi-node serving setups
- Researchers benchmarking NVFP4 models with draft-model acceleration
Use cases
- Serve Inkling-Small-NVFP4 with DSpark speculative decoding behind an OpenAI-compatible endpoint
- Reproduce the C1-C5 benchmark on two DGX Spark nodes
- Use the provided mods and recipe as a template for other EAGLE/DSpark-compatible models
- Run the included agent parser regression test after vLLM runtime updates
Strengths
- Pins the upstream runtime commit, image digest, target model, and draft model commit for reproducibility
- Measured sustained aggregate throughput up to 61.9728 output tok/s at C5 with median TTFT 0.5845 s
- Includes vLLM code modifications that expose the hidden states required by the DSpark draft while preserving the fused path
- Documents the C6 failure explicitly rather than presenting it as a successful result
Considerations
- C6 concurrency reproducibly fails with an engine watchdog RPC timeout in sample_tokens, so the stable supported profile is capped at C5
- FP8 and NVFP4 KV caches are not enabled; CUDA graph capture is also disabled due to an illegal memory access, so the verified configuration uses eager execution with BF16/auto KV cache
- No repository license has been selected, and weights, caches, credentials, SSH keys, and host-specific config.env files are excluded
README quick start
Quick start
Description
Inkling Small NVFP4 on 2x DGX Spark with TP2, 262K context, BF16 KV cache, and DSpark speculative decoding.
Related repositories
Similar projects matched by category, topics, and programming language.
talivia-group
Featured
talivia
Talivia is an open-source, self-hosted analytics platform that combines web analytics, session replay, and customer revenue tracking from multiple payment providers.
lopopolo
Featured
harness-engineering
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.