Pekingman is a multimodal agent system that integrates perception, long-term memory, human-like reasoning, emotional consistency, and real-time action to create believable NPC behavior.

Stars

933

7-day growth

No data

Forks

43

Open issues

0

License

No data

Last updated

2026-07-07

Why it is worth attention

It combines environmental sensing, persistent memory, humanized reasoning, and emotional expression into a single continuous agent loop, enabling NPCs with coherent and lifelike behavior across long interactions.

Who it is for

  • Game developers building open-world or simulation games
  • Metaverse and digital human platform creators
  • AI researchers studying embodied agents and memory systems
  • Developers of interactive virtual characters for commercial content

Use cases

  • Powering NPCs in large open-world titles with multimodal perception and memory
  • Creating digital humans that maintain long-term interaction history and personality
  • Simulating believable agents in high-complexity virtual environments
  • Building interactive characters for metaverse platforms that need context-aware responses

Strengths

  • Long-term memory iteration that persists and references historical interactions
  • Humanized reasoning and emotional output for consistent character behavior
  • Multimodal environmental sensing from multiple signal types
  • Real-time response logic that matches current context

Considerations

  • Several advanced capabilities are experimental and may fall back when optional services are unavailable
  • Dependencies for optional perception and generation may require platform-specific installation
  • Advanced demos require a modern GPU; real-time 3D simulations need a capable workstation

README quick start

Pekingman v1.0.0: A Multimodal AGI-Oriented Agent System

TingYun Yin, Massachusetts Institute of Technology (MIT)    Peixu Cai, University of Southern California (USC)    Shahzaib Saqib Warraich, University of Southern California (USC)    Osama Fawad, Université Bourgogne Europe (UBE)

Project Page | Demo Video

Pekingman v1.0.0 is a multimodal AGI-oriented agent system designed for environmental perception, long-term memory, human-like reasoning, emotional consistency, and real-time behavioral response. Equipped with a high-capacity persistent memory system, it can store, retrieve, and revisit historical events, giving NPCs authentic human-like behavioral logic.

Abstract

Pekingman connects perception, memory, reasoning, emotion, and action into one continuous agent loop. The system is designed for virtual characters that need to observe changing environments, remember past interactions, make context-aware decisions, and respond with consistent behavior.

The current implementation focuses on:

  • ingesting multi-source environmental signals;
  • extracting emotional and semantic context;
  • storing and revisiting historical interactions;
  • selecting behavior from live context;
  • producing responsive actions for interactive NPC scenarios.

Human-Equivalent Capabilities

Pekingman is designed to replicate a broad set of core cognitive and behavioral capabilities associated with human-like agents. It continuously senses surrounding environmental changes, generates context-matched behavioral feedback to external stimuli, and maintains long-term memory of past experiences. It also supports humanized logical reasoning, sequential decision-making, and consistent emotional tendency expression, enabling NPCs to exhibit coherent, lifelike behaviors across long-running interactions.

The core capability set includes:

  1. Environmental sensing: continuously observe multi-dimensional scene changes through multimodal input.
  2. Real-time response logic: generate natural feedback and action plans matched to current external conditions.
  3. Long-term memory iteration: persist, retrieve, and reference historical interactions to maintain sustained character consistency.
  4. Humanized decision and emotion output: simulate human-style reasoning, preference shifts, task choices, and emotional expression.

This makes

Related repositories

Similar projects matched by category, topics, and programming language.

syabro
syabro GitHub avatar

neat-annotations

A CSS-only library for adding hand-drawn arrows and handwritten labels to any webpage using a single self-contained stylesheet, no JavaScript or build tools required.

HTML
378
thebuggeddev
thebuggeddev GitHub avatar

football-stadium

StadiView is a procedural 3D football stadium concept demo that lets users explore a fully generated stadium, select seats, and preview the view from each seat.

HTML
349
xiaohuailabs
xiaohuailabs GitHub avatar

xiaohu-ip-studio

Xiaohu IP Studio is an open-source Agent Skill that provides a Chinese article illustration methodology with 31 hand-drawn original IP characters, enabling AI agents to automatically generate consistent, character-driven illustrations for articles.

HTML
254