It provides a structured, multi-modal evaluation across diverse office professions, using strict rubric-based scoring and an open leaderboard, making it a rigorous test for agent capabilities in practical scenarios.
FORTE (Full-cycle Office Real-world Task Evaluation) is a
general agent benchmark for evaluating AI agents on daily office productivity
across 15 corporate professions. Each
task ships as a single .md file with task description plus a multi-modal office environment (xlsx / docx /
pptx / pdf / csv / images / source files), and is scored by an LLM judge
that checks the agent's workspace artifacts against a set of expert-annotated rubrics.
A Chinese version of this README is also available.
Dataset: The full benchmark contains 180 tasks, with at least 10 tasks per profession across all 15 corporate professions. This repository ships one demo task per profession, bundled with the input files, skills, and rubrics the evaluation depends on. The SRE profession's skills are not included because they depend on internal gateways. In addition to the profession-specific skills shipped with each task, we recommend installing a set of general-purpose skills (e.g., skills that assist with reading and editing office documents) to further improve agent performance. For example, the Anthropic skills repository provides high-quality general skills such as docx, pptx, xlsx, and pdf.
Leaderboard: See the FORTE Leaderboard, evaluated on the full 180-task dataset.
| Dimension | FORTE |
|---|---|
| Task expression | Single .md + YAML frontmatter; sections ## Prompt, ## Grading Criteria |
| Modalities | Text + xlsx + docx + pptx + pdf + csv + images |
| Professions | Marketing, Sales, BA, Operations, Dev, SRE, HR, Finance, PM, Legal, Algorithm, QA, UI/UX, Administration, and General |
| Grading | LLM-as-judge, all-or-nothing scoring (i.e., score = 1 iff every rubric item passes) |
| Metrics | Avg@3, Pass@3, Pass^3 |
| Agent runtime | OpenClaw inside a single Docker image |
| Requirements | Python 3.10+ stdlib + `d |
FORTE (Full-cycle Office Real-world Task Evaluation) is a general agent benchmark for evaluating AI agents on daily office productivity across 15 corporate professions.
Similar projects matched by category, topics, and programming language.
Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.
A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.
mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.
