slvDev
精选
esp32-ai
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
AI 仓库情报摘要
FR-AI / ANALYSIS为什么值得关注
它没有使用单一描述分数，而是把覆盖率和精确率分开度量，同时提供公开数据集、评测代码、排行榜和下游评测集成，是描述感知模型评估的实用资源。
适合谁使用
- 从事视觉语言模型评估的研究人员
- 开发图像描述或文生图系统的工程师
- 需要清单式指标的基准测试维护者
- 对比 VLM 理解与 T2I 生成质量的研究者
典型使用场景
- 对自定义描述结果计算 Coverage 和 Precision 分数
- 运行内置描述生成与打分流程，比较不同描述模型
- 结合 README 中的下游工具评估文生图模型
- 在完整清单评测前，先校验提交 JSON 格式
项目优势
- 指标解耦清晰：C=100*(yes+no)/total，P=100*yes/(yes+no)
- 清单数据集规模较大：300 张图像、14,965 条原子清单项，覆盖四个类别
- 提供内置描述生成、评分和流水线脚本，并有 Python API
- 可对接 VLMEvalKit、GenEval、DPG-Bench 和 T2I-CompBench 进行下游分析
使用前须知
- 论文尚未正式发布，方法细节还未经过同行评审
- 默认评测模型 Qwen2.5-72B-Instruct 需要较高 GPU 资源
- 排行榜目前标注为初步结果，公开数据可能调整
README 快速开始
A Decoupled Caption Evaluation across Understanding and Generation
🏆 Leaderboard · 🤗 Hugging Face · 🔗 Downstream · 📄 Paper
A checklist-based benchmark that decouples caption quality into Coverage (C) and Precision (P) (0–100), and studies how each profile transfers to VLM understanding and T2I generation.
🔥 News
- [2026-07] 🏆 Project page & leaderboard: https://liuzhipenggg.github.io/CAPEval/
- [TBD] 📄 Paper release (arXiv).
- [2026-07] 💻 Code released: captioning, checklist judging, and C / P evaluation.
📎 Resources
- 📄 Paper: [Coming soon]
- 🌐 Project page: https://liuzhipenggg.github.io/CAPEval/
- 🤗 Hugging Face Dataset: LiuzhipengUCAS/CAPEval
- 🏆 Leaderboard: https://liuzhipenggg.github.io/CAPEval/leaderboard/
🚀 Installation
git clone https://github.com/liuzhipenggg/CAPEval.git
cd CAPEval
pip install -r requirements.txt
requirements.txt pins light deps and keeps transformers 4.x with `huggingface-hub/
caption/prompt.json
metrics/metrics.json
metrics/results_summary.json
```python
from capeval import CAPEval
score = CAPEval().evaluate_map({"SO001.jpg": "A detailed caption..."})
print(score["C"], score["P"])
print(score["per_category"])
📦 Dataset
300 images · 14,965 atomic checklist items · data/ (image/, gt_caption.jsonl, checklist.jsonl; join key img_path) · Hugging Face
Per-category C / P use the filename prefix (SO / PA / TI / DK). Optional tables in data/meta/ list category / subcategory labels (not required by the scorer).
Super-category Prefix Images Checklist items Avg. / image
Scene & ObjectSO1487,03947.6 People & ActivityPA603,16452.7 Text & InterfaceTI402,14653.6 Design & KnowledgeDK522,61650.3 Total30014,96549.9
🔗 Downstream evaluation
Paper downstream numbers use official stacks:
- Understanding: VLMEvalKit — MME, MMBench (EN/CN), ScienceQA, OCRBench, RealWorldQA, MMStar, MMVP, POPE, HallusionBench, AMBER, CV-Bench (2D/3D), HR-Bench (4K/8K), BLINK, AI2D, SEED-Bench, MMMU
- Generation: GenEval · [DPG-Bench (ELLA)](https://github.com/Tencent
项目描述
CAPEVAL: A DECOUPLED CAPTION EVALUATION ACROSS UNDERSTANDING AND GENERATION
相关仓库与替代方案
根据分类、Topic 和编程语言匹配的相似项目。
gavamedia
精选
deltafin
Deltafin is a research project that runs the 2.8-trillion-parameter Mixture-of-Experts model Kimi K3 on a single Apple Silicon Mac (e.g., M1 Max with 64 GB) at about 16 seconds per token, using exact, reproducible inference with local or streaming expert loading.
jamesob
精选
local-llm
A comprehensive guide for building and configuring a high-end local machine to run state-of-the-art LLMs, with detailed hardware choices, BIOS tuning, and Docker-based model serving.