Generative Representation Learning with Information Disentanglement and Global Alignment for Infrared and Visible Image Fusion

Paper

Pattern Recognition

GDAFusion is a deep learning framework for fusing infrared and visible images. It uses a two-phase training strategy:

• Phase I — Disentanglement Pre-training: The Mamba-based encoder learns to disentangle shared and modality-specific features via mutual information minimization (CLUB) and MMD loss.
• Phase II — Fusion Fine-tuning: The Restormer-based decoder is fine-tuned with a fusion-specific intensity + gradient loss.

Architecture Overview

                    ┌─────────────────┐
  VIS ──► Mamba ──►│  Base Feature   │──┬──► Decoder ──► Fused Image
  IR  ──► Mamba ──►│  Detail Feature │──┘
                    └────────┬────────┘
                             │
                    ┌────────▼────────┐
                    │  Mutual CLUB    │  (shared vs. specific disentanglement)
                    └─────────────────┘

Repository Structure

GDAFusion/
├── train.py              # Training entry point with argparse support
├── model/
│   ├── net.py            # Network: Mamba_Encoder, Restormer_Decoder, MMFusion, etc.
│   └── SS2D.py           # SS2D (Selective Scan 2D) core module
├── utils/
│   ├── loss.py           # Loss functions: Fusionloss, LMMD, KL divergence
│   └── metrics.py        # Evaluation metrics: EN, MI, SF, SSIM, VIFF, Qabf, etc.
├── requirements.txt
└── README.md

Requirements

• Python 3.8+
• PyTorch 2.x
• CUDA-compatible GPU (recommended)

Install dependencies:

pip install -r requirements.txt

Dataset Preparation

The code expects the MSRS and LLVIP datasets under a common data directory:

./data/
├── MSRS/
│   ├── visible/
│   │   ├── train/
│   │   └── val/
│   └── infrared/
│       ├── train/
│       └── val/
└── LLVIP/
    ├── visible/
    │   ├── train/
    │   └── val/
    └── infrared/
        ├── train/
        └── val/

Training

# Default settings
python train.py

# Custom paths and hyperparameters
python train.py \
    --data_dir ./data \
    --ckpt_dir ./checkpoints \
    --num_epochs 120 \
    --batch_size 3 \
    --lr 1e-4

Arguments