Pekingman

Pekingman v1.0.0: A Multimodal AGI-Oriented Agent System

TingYun Yin, Massachusetts Institute of Technology (MIT)    Peixu Cai, University of Southern California (USC)    Shahzaib Saqib Warraich, University of Southern California (USC)    Osama Fawad, Université Bourgogne Europe (UBE)

Project Page | Demo Video

Pekingman v1.0.0 is a multimodal AGI-oriented agent system designed for environmental perception, long-term memory, human-like reasoning, emotional consistency, and real-time behavioral response. Equipped with a high-capacity persistent memory system, it can store, retrieve, and revisit historical events, giving NPCs authentic human-like behavioral logic.

Abstract

Pekingman connects perception, memory, reasoning, emotion, and action into one continuous agent loop. The system is designed for virtual characters that need to observe changing environments, remember past interactions, make context-aware decisions, and respond with consistent behavior.

The current implementation focuses on:

• ingesting multi-source environmental signals;
• extracting emotional and semantic context;
• storing and revisiting historical interactions;
• selecting behavior from live context;
• producing responsive actions for interactive NPC scenarios.

Human-Equivalent Capabilities

Pekingman is designed to replicate a broad set of core cognitive and behavioral capabilities associated with human-like agents. It continuously senses surrounding environmental changes, generates context-matched behavioral feedback to external stimuli, and maintains long-term memory of past experiences. It also supports humanized logical reasoning, sequential decision-making, and consistent emotional tendency expression, enabling NPCs to exhibit coherent, lifelike behaviors across long-running interactions.

The core capability set includes:

1. Environmental sensing: continuously observe multi-dimensional scene changes through multimodal input.
2. Real-time response logic: generate natural feedback and action plans matched to current external conditions.
3. Long-term memory iteration: persist, retrieve, and reference historical interactions to maintain sustained character consistency.
4. Humanized decision and emotion output: simulate human-style reasoning, preference shifts, task choices, and emotional expression.

This makes