Why it is worth attention

It provides a rare end-to-end lab experience with real hardware (RNGD job server), guided optimization, and a schedule viewer, bridging low-level architecture understanding with practical model optimization.

Who it is for

AI compiler engineers
performance engineers
ML researchers interested in hardware-software co-design
Rust developers working on AI accelerators

Use cases

Learning vISA kernel programming for RNGD
Optimizing transformer kernel performance (GEMM, GEMV, attention)
Benchmarking AI accelerator efficiency with real models
Practicing kernel fusion and memory optimization techniques

Strengths

Real hardware access via RNGD job server for accurate measurements
Covers both simple kernels and a complete Qwen3 0.6B model
Includes a schedule viewer for bottleneck analysis
Guided and free optimization sessions with concrete optimization menu

Considerations

Requires x86-64 Linux environment (no native macOS/Windows/ARM support)
Needs Rust toolchain, cargo-furiosa-opt, and a job server account
Emulation is slow for full model; real runs require queue submission
Focuses solely on Furiosa RNGD, not transferable to other accelerators

README quick start

Furiosa Optimization Tutorial

Overview

This tutorial covers the hardware architecture and programming principles of Furiosa RNGD, a next-generation AI accelerator
Participants use the dedicated development environment to implement and optimize Tensor Contraction and Transformer kernels
Goal: understand the architecture, write vISA kernels, and optimize a real Qwen3 0.6B model

Prerequisites

x86-64 Linux: published artifacts are x86_64-unknown-linux-gnu only, so binstall finds nothing on macOS, Windows, or ARM
- macOS: work in an x86-64 Linux container (setup)
- Windows: work in an x86-64 Linux container (e.g. WSL2)
Rust toolchain and cargo-furiosa-opt: installed during the lecture (Section 1)
Qwen3 0.6B baseline code: provided separately
RNGD job server account: sign up with your GitHub username before the lab

Submit your GitHub username via the Google Form. Accounts on the job server are provisioned from those usernames, so the form must close before the lab starts.

Company Intro (10:00 to 10:15, 15 min)

FuriosaAI is committed to provide a programmable AI accelerator.

1. Installation and Quick Start (10:15 to 10:50, 35 min)

1.1 Installation (book: Introduction)

# 1. Rust toolchain: install rustup from https://rustup.rs/, then pin nightly
rustup toolchain install nightly-2026-05-01

# 2. cargo-furiosa-opt
cargo install cargo-binstall
cargo binstall cargo-furiosa-opt

# 3. Project scaffold
cargo install cargo-generate
cargo generate furiosa-ai/furiosa-opt base-template   # prompt: enter base-template
cd base-template

cargo-furiosa-opt is prebuilt: the installing toolchain is irrelevant, the scaffold's rust-toolchain.toml pins the build one
The Favorite ... not found in config line from cargo generate is expected
RNGD SDK and driver: required only for --backend npu
Today's lab: emulation and RNGD job server (SDK on the job server)
Job server (https://arena.furiosa.ai): get in before Section 1.3, in two steps:
- A. Sign in to the web UI: open https://arena.furiosa.ai, log in with GitHub SSO. Your account exists only if the GitHub username you submitted in the form (see Prerequisites) was registered, if the log

Related repositories

Similar projects matched by category, topics, and programming language.

7-e1even

Featured

learn-agent

A collection of notes on building coding agents, derived from the production development of the desktop agent Reina, with each mechanism simplified into a zero-dependency, single-file Node.js demo.

AI & Machine LearningLarge Language Models

218

slvDev

Featured

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models

1,960

jamesob

Featured

local-llm

A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.

AI & Machine LearningLarge Language Models

1,660