一个零依赖的 Python 工具,用于审计 AI 代理的技能目录,揭示哪些技能实际可加载且可被发现,并自动生成修复描述的计划。

Stars

5

7 天增长

暂无数据

Fork 数

0

开放 Issue

0

开源协议

MIT

最近更新

2026-07-30

AI 仓库情报摘要
FR-AI / ANALYSIS

为什么值得关注

它暴露了代理技能管理中的隐蔽问题——磁盘上多数技能是死副本或无法被检索到——并在无需 API 密钥或模型调用的前提下,给出可量化的前后对比指标。

适合谁使用

  • 维护 Claude Code、OpenClaw 或 Hermes 技能集合的开发者
  • 排查代理为何从不调用某些技能的 AI 工程师
  • 管理大型 LLM 代理工具目录的团队
  • 需要透明路由基准的 LLM 工具评估人员

典型使用场景

  • 审计技能文件夹,统计实际可加载技能并标记重复或名称遮蔽
  • 识别描述含糊或冲突的技能,这些技能导致检索失败
  • 为弱技能描述生成可供 LLM 直接使用的重写计划(rewrites.md)
  • 在修复前后度量路由准确率,确认改善效果

项目优势

  • 零运行时依赖——只需 Python 3.10+,可离线毫秒级运行
  • 与运行时无关,适用于任何加载 SKILL.md 文件的系统
  • 提供可操作的、基于证据的输出,如自检索失败率和冲突分数
  • 附带示例技能和一个记录完整、含具体指标的真实案例研究

使用前须知

  • 仅使用词法 BM25 检索;无法处理与正确技能毫无词汇重合的请求
  • 评估集不能跨机器迁移,因为每条期望都必须指向本机已安装的技能
  • 使用统计无法覆盖由 cron、钩子或插件自动加载触发的技能,因此“从未使用”不能证明技能无用
  • 插件自有技能的改写会在下次插件升级时被覆盖

README 快速开始

skill-audit-router

Your AI agent has hundreds of skills. Most of them can't be found.

This tells you which ones — and fixes them.

Works with Claude Code, OpenClaw, Hermes, or anything that loads SKILL.md files.


The idea, in plain words

Imagine your agent has a toolbox with 200 tools. Every tool has a label on it.

When you ask the agent to do something, it doesn't look inside the tools. It reads the labels and picks one.

Two things go wrong, and you can't see either of them:

🧰 Half the tools aren't actually in the box. They're in a drawer, a spare box in the garage, or they're copies of the same hammer. You think you have 200. You might have 80.

🏷️ Half the labels are unreadable. A tool labelled "Helper mode" will never be picked for "fix my leaky tap" — because nothing in the label matches what you asked for.

This tool opens the box, counts what's really there, and tells you which labels nobody can read. Then it writes you a to-do list for fixing them.



Try it in 30 seconds

No setup. No install. No API key. It ships with four example skills so you can see exactly what it does before pointing it at anything of yours.

git clone https://github.com/rushindrasinha/skill-audit-router.git
cd skill-audit-router
python3 skill_audit.py --roots examples/fixture-skills --include-workspace

That's the screenshot above. Four example skills, each teaching one lesson:

Example skillScoreWhat it shows you
good-example86 🟢Says what it makes, lists the exact words a user would type, and says what it's not for
confusable-a60 🟡Nearly identical to its twin — the tool catches the collision
confusable-b60 🟡…the other half of that pair, flagged at margin 0.038
weak-example21 🔴Labelled "Helper mode." — unfindable, and the report says exactly why

Why --include-workspace? The examples live in this repo, not in your agent's real skills folder. By default the audit ignores anything your agent can't actually load — which is the whole point. This flag says "look at them anyway."


The problem, measured

Run it on a

项目描述

Find out which of your agent's skills can actually be found, then route a query to the right one. Claude Code / OpenClaw / Hermes. Pure stdlib.

相关仓库与替代方案

根据分类、Topic 和编程语言匹配的相似项目。

lopopolo
精选
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI 与机器学习AI 智能体
2,390
MoonshotAI
精选
MoonshotAI GitHub avatar

Kimi-K3

Kimi K3 is an open-weight, 2.8T-parameter native multimodal agentic model with a 1M-token context window, designed for frontier coding, knowledge work, and reasoning tasks.

AI 与机器学习AI 智能体
3,348
mshumer
精选
mshumer GitHub avatar

Claude-of-Duty

A browser-based first-person shooter built entirely with procedural generation and orchestrated AI agents, featuring 55k lines of Three.js/WebGL2 code and no art assets.

AI 与机器学习AI 智能体
1,234