MoonshotAI
Featured
Kimi-K3
Kimi K3 is an open-weight, 2.8T-parameter native multimodal agentic model with a 1M-token context window, designed for frontier coding, knowledge work, and reasoning tasks.
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It fills a critical gap by documenting real-world issues and workarounds for consumer Blackwell hardware, where official support is incomplete, with dated and versioned entries anyone can reproduce.
Who it is for
- RTX 50-series GPU owners
- AI developers running local inference or fine-tuning on consumer hardware
- Hobbyists and researchers building single-GPU stacks
- Engineers debugging PyTorch/CUDA compatibility on Blackwell
Use cases
- Rapidly diagnosing and fixing torch kernel mismatches on sm_120
- Optimizing inference throughput and memory for large models on a single 32GB card
- Quantizing models (FP4, AWQ, GGUF) to fit VRAM while measuring real performance
- Building AI frameworks from source with correct CUDA and toolkit settings
Strengths
- All claims are dated, versioned, and reproducible using open code and data
- Covers a wide range of practical issues: torch, inference, quantization, source builds, modalities
- Provides a simple 5-minute kill test to quickly validate a stack before deeper investment
- Based entirely on a real, purchasable card with no cloud dependencies
Considerations
- Focused on a single RTX 5090; multi-GPU or datacenter scenarios are not covered
- Entries may become stale as software versions advance; the 'Verified' line provides trust guidance
- Not a general guide for all GPUs—only consumer Blackwell (sm_120) issues are addressed
README quick start
the sm_120 field guide
AI on consumer Blackwell. the fixes and footguns from one RTX 5090, measured honestly.
every entry here is something I hit running modern AI on an RTX 50-series card: the error message, the mechanism behind it, the fix that worked, and the date + versions it was verified on. if I didn't run it, it isn't here. if you own a 50-series card (sm_120), this is the debugging session you get to skip.
most AI infrastructure targets datacenter cards. consumer Blackwell is close enough to look supported and different enough to break in quiet, expensive ways: wheels that ship without kernels, and headline speedups that assume an H200. this guide maps where it breaks and what to do about it.
the machine
| part | spec |
|---|---|
| GPU | RTX 5090 32GB (Blackwell, sm_120) |
| CPU | AMD Ryzen 5 9600 (6c/12t) |
| RAM | 64GB DDR5-5600 |
| OS | Ubuntu Server 26.04 LTS, headless |
one card you can actually buy. no rack, no cloud fallback.
the rules
- reproducible. open code, open data, on a machine I specify. the raw benchmark data behind these entries is public.
- honest. every claim is dated and versioned. entries go stale as stacks move; the
Verified:line tells you how much to trust each one. - sovereign. if a result needs the cloud, it doesn't count here.
start here: the 5-minute kill test
before building any repo's stack on a 50-series card, test only its torch pin:
uv venv /tmp/probe && source /tmp/probe/bin/activate
uv pip install "torch==" --index-url https://download.pytorch.org/whl/cu124
python -c 'import torch; x = torch.ones(2, device="cuda"); print(x + x)'
if that fails with CUDA error: no kernel image is available for execution on the device, stop. torch wheels below 2.7 (the cu124/cu126 era) ship no sm_120 kernels, and nothing built on top of that pin can fix it. reimplement on torch ≥ 2.7 + cu128 instead. old-transformers-on-new-torch is salvageable; old-torch never is. full entry in torch, kernels & training stacks.
the guide
80 entries across seven files, newest first in each.
torch, kernels & training stacks · 7 entries
- torch pins below 2.7 ship no sm_120 kernels: kill a doomed stack in 5 minutes
- custom trust_remote_code models inherit transformers version skew: pin the OLD transformers, keep the NEW torc
Description
AI on consumer Blackwell — a field guide for the RTX 50-series (sm_120). Fixes, footguns, and honest measurement from one RTX 5090.
Related repositories
Similar projects matched by category, topics, and programming language.
xuchonglang
Featured
investing-for-beginners
A structured investing guide for Chinese beginners covering US stocks, options, and cryptocurrency, with focus on foundational concepts and risk awareness.
Krishnagangwal
Featured
CS-Fundamentals
A curated collection of Computer Science fundamentals (PDFs, notes, cheatsheets, interview question banks) for placement preparation, covering seven core subjects plus general resources.