AptAvatar is a fast, vivid, open-source system for generating long-form audio-driven avatar videos using only 2 sampling steps, designed for production use.

Stars

7

7-day growth

No data

Forks

0

Open issues

0

License

Apache-2.0

Last updated

2026-07-28

AI repository intelligence
FR-AI / ANALYSIS

Why it is worth attention

It achieves state-of-the-art results on internal benchmarks with only 2 NFE (neural function evaluations), dramatically reducing inference cost while maintaining high audio-visual alignment and video quality, and is released under Apache 2.0.

Who it is for

  • AI video generation researchers
  • avatar and digital human developers
  • streaming and content creation engineers
  • open-source video generation enthusiasts

Use cases

  • Real-time or near-real-time talking-head video generation for live streaming
  • Long-form avatar-based video production for social media or virtual presenters
  • Research into efficient diffusion distillation for audio-conditioned video
  • Rapid prototyping of personalized digital avatars driven by speech

Strengths

  • Only 2 sampling steps (NFE) vs. 4-120 for competing methods, enabling faster generation
  • Outperforms many 4-step and 8-step methods across all reported metrics on internal 720p benchmark
  • Supports structured prompts with explicit frame-by-frame motion control for fine-grained generation
  • Released under permissive Apache 2.0 license with complete inference code and model weights

Considerations

  • Model weights and full benchmarks are not yet released (only inference code is available at time of writing)
  • Audio encoder is specifically Chinese wav2vec2, which may limit multilingual performance without adaptation
  • Requires FlashAttention and specific PyTorch/CUDA versions, potentially limiting hardware compatibility

README quick start

AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

-->

Links are placeholders for now and will be updated as project pages, model cards, community channels, and reports become available.

✨ Key Features

  • Endpoint-Anchored Distribution Distillation: It augments vanilla distribution matching with a dedicated Anchor Score Estimator trained on the trajectory-endpoint distribution defined from a frozen pretrained 4-step bridge generator. This provides an attainable endpoint-level anchor for the evolving two-step student.
  • Self-Generated History Replay: It reuses cached outputs from earlier generator checkpoints as history conditions during chunk-wise training. This approximates inference-time self-conditioning, mitigating quality degradation from accumulated history errors

🔥 News

  • 2026.06.29 - Released the initial AptAvatar repository with open-source inference code.

📋 Todo

  • Technical report
  • Project page
  • Inference code
  • Checkpoint release
  • Additional demos

🧩 Pipeline

🎬 Examples

🚀 Quick Start

🛠️ Installation

Create and activate the environment:

conda create -n aptavatar python=3.10
conda activate aptavatar

Install PyTorch:

pip install torch==2.8.0 torchvision==0.23.0 --index-url https://download.pytorch.org/whl/cu128

Install project dependencies:

pip install -r requirements.txt

Install FlashAttention:

pip install ninja
pip install flash_attn==2.8.0.post2 --no-build-isolation

[!NOTE] If flash_attn_3 is installed in your environment, please uninstall it, as it conflicts with the FlashAttention-3 kernels bundled in xformers:

pip uninstall flash_attn_3

Install FFmpeg:

# Ubuntu / Debian
apt-get install ffmpeg

# CentOS / RHEL
yum install ffmpeg ffmpeg-devel

or install through conda:

conda install -c conda-forge ffmpeg==7

📦 Model Preparation

The current repository provides inference code only. After checkpoints are released, place them under the following paths:

ComponentExpected PathStatus
AptAvatar checkpointmodels/AptAvatarAptAvatar
Audio encoder`models/chine

Description

Official Implementation of AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatars

Related repositories

Similar projects matched by category, topics, and programming language.

lopopolo
Featured
lopopolo GitHub avatar

harness-engineering

Harness Engineering is a methodology for improving coding agent outputs by carefully crafting the environment around them—providing curated context, tools, and executable constraints that encode an organization’s nonfunctional requirements and cumulative lessons.

AI & Machine LearningAI Agents
2,390
slvDev
Featured
slvDev GitHub avatar

esp32-ai

A 28.9 million parameter language model runs on an $8 ESP32-S3 microcontroller entirely on-device, generating simple stories at about 9.5 tokens per second.

AI & Machine LearningLarge Language Models
1,960
littledivy
Featured
littledivy GitHub avatar

mimic

mimic captures traffic from any iOS or web app and automatically generates a Python client library that lets you call the app's API like a regular library.

AI & Machine Learning
1,482