🚀 Multi Scale Efficient Global Context Vision Transformer

Multi-Scale Efficient Global Context Vision Transformer (MS-EffGCViT) is a hybrid CNN-ViT architecture for deepfake detection. It fuses CNN-driven spatial inductive bias with hierarchical global-context attention to catch both local artifacts (textures, blending seams) and global artifacts (lighting, structural inconsistency).

A single architecture ships in two sizes and three domain-tuned checkpoints, working on both static images and video at the frame level.

🤗 Live Demo

⚙️ Model Specifications

🧬 Model Zoo

🕐 Model Inference Latency

Latency is measured with batch size 1 on a single Tesla T4 and video (num_frames=20).

📚 Dataset: FaceForensics++

Learning to detect manipulated facial images [Paper] [Download], featuring 1,000 original YouTube videos manipulated by 5 face forgery methods.

• Source: 1,000 original videos (from 977 YouTube videos)
• Manipulation Methods: 5 types (Deepfakes, Face2Face, FaceSwap, FaceShifter, NeuralTextures)
• Faces: Trackable, mostly frontal, no occlusion

📎 Available at GitHub or Kaggle

📈 Test Evaluation

Trained and tested on the same dataset.

📊 Comparison with SOTA Models

📈 Cross-Dataset Evaluation (Trained on FaceForensics++)

Generalization to unseen domains — trained on FaceForensics++

🚀 Model Usage

pip install "deepguard>=2.0.0"

from transformers import pipeline

🖼️ Image Classification

clf = pipeline(
    "image-classification",
    model="KoreaPeter/ms-eff-gcvit-deepfake-b0-ff-plus-plus", 
    trust_remote_code=True,
)

# ── Basic Inference ───────────────────────────────────────────────
result = clf("face.jpg")
# [{'label': 'fake', 'score': 0.9712}, {'label': 'real', 'score': 0.0288}]

# ── Custom Parameters ─────────────────────────────────────────────
result = clf(
    "face.jpg",
    margin_ratio=0.2,      # Margin ratio around the detected face bbox (default: 0.2)
    conf_thres=0.5,        # Confidence threshold for YOLO face detection (default: 0.5)
    min_face_ratio=0.01,   # Minimum face-to-frame area ratio to process (default: 0.01)
    tta_hflip=0.0,         # Probability of horizontal flip for TTA (default: 0.0)
    top_k=1,               # Number of top labels to return (default: all)
)
# [{'label': 'fake', 'score': 0.9712}]

🎬 Video Classification

clf = pipeline(
    "video-classification",
    model="KoreaPeter/ms-eff-gcvit-deepfake-b0-ff-plus-plus", 
    trust_remote_code=True,
)

# ── Basic Inference ───────────────────────────────────────────────
result = clf("video.mp4")
# [{'label': 'fake', 'score': 0.9634}, {'label': 'real', 'score': 0.0366}]

# ── Custom Parameters ─────────────────────────────────────────────
result = clf(
    "video.mp4",
    num_frames=20,              # Number of frames to sample (default: 20)
    margin_ratio=0.2,           # Margin ratio around the detected face bbox (default: 0.2)
    conf_thres=0.5,             # Confidence threshold for YOLO face detection (default: 0.5)
    min_face_ratio=0.01,        # Minimum face-to-frame area ratio to process (default: 0.01)
    tta_hflip=0.0,              # Probability of horizontal flip for TTA (default: 0.0)
    agg_mode="conf",            # Aggregation mode: 'conf' | 'mean' | 'vote' (default: 'conf')
    return_frame_scores=True,   # Return per-frame scores (default: False)
)
# [{'label': 'fake', 'score': 0.9634},
#  {'label': 'real', 'score': 0.0366},
#  {'frame_scores': [0.97, 0.95, 0.98, ...], 'agg_mode': 'conf'}]

Deep Dive into Model

Part 1: CNN-based Patch Embedding for Spatial Inductive Bias

While traditional Vision Transformers (ViTs) utilize a Linear Projection for patch embedding, our proposed model adopts a CNN-based Patch Embedding module incorporating MBConvBlocks.

• Injecting Inductive Bias : Standard ViTs often suffer from a lack of inherent spatial inductive bias, typically necessitating massive datasets to learn fundamental visual structures from scratch. In contrast, our CNN-based module leverages overlapping receptive fields to facilitate information sharing between neighboring patches. By explicitly injecting this spatial bias into the architecture, the model achieves more stable and accelerated convergence during the training process.

Part 2: Long-Short Range Spatial Interaction

We utilizes two distinct types of self-attention to capture both long-range and short-range information across feature maps.

• Local Window Attention: this model efficiently captures local textures and precise spatial details while maintaining linear computational complexity relative to the image size.

• Global Window Attention: Unlike Swin Transformer, this module utilizes global-queries that interact with local window keys and values. This allows each local region to incorporate global context, effectively capturing long-range dependencies and providing a comprehensive understanding of the entire spatial structure

Part 3: Computational Efficiency

• Efficient Backbone While both Xception and EfficientNet show great results on DeepFake benchmarks, EfficientNet is chosen for its superior computational efficiency. By utilizing MBconv (Inverted Residual Blocks) and depthwise convolutions, it achieves significantly lower FLOPS compared to Xception.

• Window-based Attention: Instead of applying self-attention on raw images, this model operates on feature maps extracted from backbone blocks. By partitioning these maps into windows, the $O(N^2)$ complexity is restricted to the window size, siginificantly lowering the computational footprint.

Part 4: Multi-Scale Feature Map Fusion

Modern DeepFakes can leave very localized forgery region. To Capture this, we adopts a multi-scale strategy by extracting features from different levels of the backbone.

• (Subtle Artifacts): High-Resolution feature maps are extracted from early backbone blocks(l_block_idx) to capture like skin texture or boundary artifacts

• (Global Features): Low-Resolution feature maps are extracted from deeper blocks(h_block_idx) to analyze overall lighting, shadows, and structural consistency.

• Feature Fusion: The Outputs from both branches (L-GCViT and H-GCViT) are fused to make a comprehensive decision based on both local and global context.

🤝 Citation

@misc{deepguard2026,
  title  = {DeepGuard: Multi-Scale Efficient Global Context Vision Transformer for Deepfake Detection},
  author = {seoyunje},
  year   = {2026},
  url    = {https://github.com/HanMoonSub/DeepGuard}
}