为什么值得关注

通过互信息最小化实现信息解缠与全局对齐，将 Mamba 编码器与 Restormer 解码器相结合，在融合质量上达到领先水平。

适合谁使用

多模态图像融合研究人员
从事安防、夜视应用的计算机视觉工程师
探索基于 Transformer 融合架构的深度学习实践者

典型使用场景

利用红外信息增强低光照可见光图像
改善自动驾驶在恶劣环境下的目标检测
为安防监控系统生成高质量融合图像

项目优势

清晰的两阶段训练流水线，分离解缠与融合目标
采用现代架构（Mamba 和 Restormer），文献基础扎实
提供标准评估指标（EN、MI、SF、SSIM 等），便于可重复性基准测试
开源代码，包含详细的训练参数与数据集准备说明

使用前须知

仅基于 MSRS 和 LLVIP 数据集验证，对其他模态的泛化能力未知
需要兼容的 GPU 和 PyTorch 2.x，可能限制在旧硬件上的部署
未提供预训练权重，用户需从头训练指定数据集

README 快速开始

Generative Representation Learning with Information Disentanglement and Global Alignment for Infrared and Visible Image Fusion

Paper

Pattern Recognition

GDAFusion is a deep learning framework for fusing infrared and visible images. It uses a two-phase training strategy:

Phase I — Disentanglement Pre-training: The Mamba-based encoder learns to disentangle shared and modality-specific features via mutual information minimization (CLUB) and MMD loss.
Phase II — Fusion Fine-tuning: The Restormer-based decoder is fine-tuned with a fusion-specific intensity + gradient loss.

Architecture Overview

                    ┌─────────────────┐
  VIS ──► Mamba ──►│  Base Feature   │──┬──► Decoder ──► Fused Image
  IR  ──► Mamba ──►│  Detail Feature │──┘
                    └────────┬────────┘
                             │
                    ┌────────▼────────┐
                    │  Mutual CLUB    │  (shared vs. specific disentanglement)
                    └─────────────────┘

Repository Structure

GDAFusion/
├── train.py              # Training entry point with argparse support
├── model/
│   ├── net.py            # Network: Mamba_Encoder, Restormer_Decoder, MMFusion, etc.
│   └── SS2D.py           # SS2D (Selective Scan 2D) core module
├── utils/
│   ├── loss.py           # Loss functions: Fusionloss, LMMD, KL divergence
│   └── metrics.py        # Evaluation metrics: EN, MI, SF, SSIM, VIFF, Qabf, etc.
├── requirements.txt
└── README.md

Requirements

Python 3.8+
PyTorch 2.x
CUDA-compatible GPU (recommended)

Install dependencies:

pip install -r requirements.txt

Dataset Preparation

The code expects the MSRS and LLVIP datasets under a common data directory:

./data/
├── MSRS/
│   ├── visible/
│   │   ├── train/
│   │   └── val/
│   └── infrared/
│       ├── train/
│       └── val/
└── LLVIP/
    ├── visible/
    │   ├── train/
    │   └── val/
    └── infrared/
        ├── train/
        └── val/

Training

# Default settings
python train.py

# Custom paths and hyperparameters
python train.py \
    --data_dir ./data \
    --ckpt_dir ./checkpoints \
    --num_epochs 120 \
    --batch_size 3 \
    --lr 1e-4

Arguments

Argument	Default	Description
`--data_dir`	`./data`	Root directory containing MSRS/ and L

项目描述

[Pattern Recognition 2026] Generative Representation Learning with Information Disentanglement and Global Alignment for Infrared and Visible Image Fusion

GDAFusion