SEED is a self-evolving on-policy distillation framework for long-horizon LLM agents that uses a single policy checkpoint to collect trajectories, analyze them into hindsight skills, and distill skill-induced action probability shifts back into itself during RL.

Stars

198

7-day growth

No data

Forks

7

Open issues

1

License

MIT

Last updated

2026-07-17

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It introduces a novel paradigm where the policy and its hindsight supervision co-evolve, achieving dense token-level supervision from sparse trajectory rewards without external skill banks or retrieval at inference.

Who it is for

  • Researchers in reinforcement learning for LLM agents
  • Engineers building long-horizon autonomous agent systems
  • AI practitioners interested in self-improving policy distillation
  • Academics studying skill acquisition in language models

Use cases

  • Training agents for web shopping tasks (WebShop)
  • Training agents for interactive household tasks (ALFWorld)
  • Training agents for search-based open-domain question answering
  • Training agents for game-like environments such as Sokoban and EZPoints

Strengths

  • Consistently outperforms outcome-only RL (GRPO) by converting episodic rewards into token-level learning signals
  • Internalized skills during training outperform prompting with skills at inference
  • Self-evolving distillation maintains alignment with the policy's changing behaviors, outperforming static distortion
  • Demonstrated strong results across three diverse benchmarks (ALFWorld, SearchQA, WebShop)

Considerations

  • Requires two-stage training (SFT plus RL) and multiple environment-specific installations (e.g., WebShop requires Python 3.10)
  • Initial hindsight-skill annotation depends on an external LLM API (OpenAI) for the paper's workflow
  • Computational cost may be high due to on-policy distillation and simultaneous trajectory analysis

README quick start

Installation

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482