AptAvatar 是一个快速、生动的开源系统,仅需 2 步采样即可生成由音频驱动的长视频虚拟形象,专为生产环境设计。

Stars

7

7 天增长

暂无数据

Fork 数

0

开放 Issue

0

开源协议

Apache-2.0

最近更新

2026-07-28

AI 仓库情报摘要
FR-AI / ANALYSIS

为什么值得关注

它在内部基准测试中仅用 2 次神经函数评估(NFE 即实现了最先进的结果),大幅降低推理成本,同时保持高音视频对齐和视频质量,并以 Apache 2.0 协议开源。

适合谁使用

  • AI 视频生成研究人员
  • 虚拟形象与数字人开发者
  • 直播与内容创作工程师
  • 开源视频生成技术爱好者

典型使用场景

  • 直播场景下接近实时的说话头视频生成
  • 用于社交媒体或虚拟主持人的长视频虚拟形象制作
  • 音频条件视频生成的高效扩散蒸馏研究
  • 基于语音驱动的个性化数字虚拟形象快速原型开发

项目优势

  • 仅需 2 步采样(NFE),而竞品需 4-120 步,显著提升生成速度
  • 在内部 720p 基准测试中,所有指标均优于多数 4 步和 8 步方法
  • 支持结构化提示,可精确描述每帧运动,实现细粒度控制
  • 采用宽松的 Apache 2.0 许可证,提供完整推理代码和模型权重

使用前须知

  • 目前仅发布推理代码,模型权重和完整基准尚未提供
  • 音频编码器为中文 wav2vec2,可能限制多语言性能
  • 依赖特定 PyTorch/CUDA 版本和 FlashAttention,硬件兼容性有限

README 快速开始

AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

-->

Links are placeholders for now and will be updated as project pages, model cards, community channels, and reports become available.

✨ Key Features

  • Endpoint-Anchored Distribution Distillation: It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student.
  • Self-Generated History Replay: It reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time self-conditioning, mitigating quality degradation from accumulated history errors

🔥 News

  • 2026.06.29 - Released the initial AptAvatar repository with open-source inference code.

📋 Todo

  • Technical report
  • Project page
  • Inference code
  • Checkpoint release
  • Additional demos

🧩 Pipeline

🎬 Examples

🚀 Quick Start

🛠️ Installation

Create and activate the environment:

conda create -n aptavatar python=3.10
conda activate aptavatar

Install PyTorch:

pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128

Install project dependencies:

pip install -r requirements.txt

Install FlashAttention:

pip install ninja
pip install flash_attn==2.8.0.post2 --no-build-isolation

[!NOTE] If flash_attn_3 is installed in your environment, please uninstall it, as it conflicts with the FlashAttention-3 kernels bundled in xformers:

pip uninstall flash_attn_3

Install FFmpeg:

# Ubuntu / Debian
apt-get install ffmpeg

# CentOS / RHEL
yum install ffmpeg ffmpeg-devel

or install through conda:

conda install -c conda-forge ffmpeg==7

📦 Model Preparation

The current repository provides inference code only. After checkpoints are released, place them under the following paths:

ComponentExpected PathStatus
AptAvatar checkpointmodels/AptAvatarAptAvatar
Audio encoder`models/chine

项目描述

Official Implementation of AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

相关仓库与替代方案

根据分类、Topic 和编程语言匹配的相似项目。

lopopolo
精选
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI 与机器学习AI 智能体
2,390
slvDev
精选
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI 与机器学习大语言模型
1,960
littledivy
精选
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI 与机器学习
1,482