slvDev
Featured
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
AI repository intelligence
FR-AI / ANALYSISWhy it is worth attention
It provides a structured, reproducible way to measure and compare LLM quality, with flexible options for language, scale, and weighting.
Who it is for
- LLM developers and engineers
- AI researchers evaluating model outputs
- Product teams building agent evaluation systems
- QA professionals creating scoring rubrics
Use cases
- Grading LLM responses on custom criteria
- Comparing model performance in benchmarks
- Setting up evaluation rubrics for agent workflows
- Creating repeatable grading templates for automated testing
Strengths
- Supports multilingual rubrics (en, zh, bilingual)
- Offers configurable scales (1-5, 0-2, pass-fail) and weighting
- Encourages reproducibility and comparability of evaluations
- Integrates via simple CLI install (npx skills add)
Considerations
- Full workflow details reside in external SKILL.md, not the README alone
- Scope is limited to rubric design, not execution of evaluations
- Requires familiarity with the 'skills' ecosystem for installation
README quick start
Eval Rubric
Design evaluation rubrics for LLM/agent outputs: criteria, scores, failure modes, examples.
中文简介： 你做可复现、可对比的模型输出评测表。 — 详见 README.zh.md.
What this skill does
- You build rubrics that make model quality measurable and comparable
- Scope: See SKILL.md vertical lock.
- Deliverable: See SKILL.md deliverable.
- Triggers: eval rubric, LLM eval, grading rubric, agent eval, 评测量表.
Install
After this package is on GitHub:
npx skills add / --skill eval-rubric
Local monorepo: point your agent at skills/eval-rubric/ (this folder).
Options
| Option | Values | Default |
|---|---|---|
lang | en · zh · bilingual | en |
scale | 1-5 · 0-2 · pass-fail | 1-5 |
weighting | equal · custom | equal |
Runtime output language follows lang (default en). Skill docs stay EN-primary.
When to use
| Use | Don't |
|---|---|
| (see bullets) | (see bullets) |
Use
- When you need
eval-rubricworkflow
Don't
- Out-of-scope tasks listed in SKILL.md
Deliverable
See SKILL.md deliverable.
Full workflow, hard rules, and QA: see SKILL.md.
Example
Monorepo sample (if present): ../../output/samples/eval-rubric-example-01.md
License
MIT — see LICENSE.
Related repositories
Similar projects matched by category, topics, and programming language.
jamesob
Featured
local-llm
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.
makecindy
Featured
cindy
Cindy is an open-source AI agent that runs locally on your machine, integrates multiple AI harnesses and models, and provides memory, skills, and automation to perform real work in your projects and apps.