KnowAct-GUIClaw is an open-source personal GUI assistant framework that uses a Know–Route–Act–Reflect loop with self-evolving memory and skill to achieve state-of-the-art performance on mobile and desktop automation across Android, iOS, HarmonyOS, and desktop platforms.

Stars

482

7-day growth

No data

Forks

42

Open issues

0

License

MIT

Last updated

2026-07-27

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It achieves a state-of-the-art 64.1% success rate on the long-horizon MobileWorld benchmark, outperforming both open and closed agents like GPT-5.5 and Seed-2.0-Pro, and generalizes to multiple base models with significant improvements (+8.5% on Kimi-2.6, +16.2% on Qwen3.5-35B-A3B).

Who it is for

  • GUI automation researchers and practitioners
  • Mobile and desktop app testers and QA engineers
  • AI agent developers building personal assistants
  • Open-source community interested in autonomous UI agents

Use cases

  • Automating repetitive tasks on Android devices (e.g., open Settings, enable Wi-Fi)
  • Cross-platform GUI testing and validation using dry-run mode
  • Building a personal assistant that learns from user interactions and remembers policies
  • Skill extraction and shortcut validation from Android manifests for faster task execution

Strengths

  • State-of-the-art benchmark results on MobileWorld (64.1%) with open-source models
  • Multi-platform support: Android (ADB), iOS (WebDriverAgent), HarmonyOS (HDC), desktop (local)
  • Self-evolving memory (policy.md, gui_memory_bank.jsonl) and skill (skills.py) that improve over time
  • Flexible deployment as a complete agent host (nanobot) or standalone CLI tool with adapter patterns

Considerations

  • Requires Python 3.11+, uv, a multimodal vision model, and appropriate backend tools (ADB, WebDriverAgent, etc.)
  • Skill extraction and memory induction are disabled by default; dry-run mode recommended to prevent unintended device changes
  • Policy rules in memory are model guidance, not enforced—OS, backend, or sandbox controls needed for mandatory restrictions

README quick start

Quick start: complete host

Description

Recursive Self-Improvement GUI Assistant

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482