Qwen-Image-Lora-Faceseg

Model description

Face Segmentation Model Description

Overview

This is a LoRA fine-tuned face segmentation model based on Qwen-VL (Qwen Vision-Language) architecture, specifically designed to transform facial images into precise segmentation masks. The model leverages the powerful multimodal capabilities of Qwen-VL and enhances it through Parameter-Efficient Fine-Tuning (PEFT) using LoRA (Low-Rank Adaptation) technique.

Model Architecture

• Base Model: Qwen-Image-Edit (built on Qwen-VL foundation)
• Fine-tuning Method: LoRA (Low-Rank Adaptation)
• Task: Image-to-Image translation (Face → Segmentation Mask)
• Input: RGB facial images
• Output: Binary/grayscale segmentation masks highlighting facial regions

Training Configuration

• Dataset: 20 carefully curated face segmentation samples
• Training Steps: 900-1000 steps
• Prompt: "change the image from the face to the face segmentation mask"
• Precision Options:
  • BF16 precision for high-quality results
  • FP4 quantization for memory-efficient deployment

Key Features

1. High Precision Segmentation: Accurately identifies and segments facial boundaries with fine detail preservation
2. Memory Efficient: FP4 quantized version maintains competitive quality while significantly reducing memory footprint
3. Fast Inference: Optimized for real-time applications with 20 inference steps
4. Robust Performance: Handles various lighting conditions and facial orientations
5. Parameter Efficient: Only trains LoRA adapters (~1M parameters) while keeping base model frozen

Technical Specifications

• Inference Steps: 20
• CFG Scale: 2.5
• Input Resolution: Configurable (typically 512x512)
• Model Size: Base model + ~1M LoRA parameters
• Memory Usage:
  • BF16 version: Higher memory, best quality
  • FP4 version: 75% memory reduction, competitive quality

Use Cases

• Identity Verification: KYC (Know Your Customer) applications
• Privacy Protection: Face anonymization while preserving facial structure
• Medical Applications: Facial analysis and dermatological assessments
• AR/VR Applications: Real-time face tracking and segmentation
• Content Creation: Automated face masking for video editing

Performance Highlights

• Accuracy: Significantly improved boundary detection compared to base model
• Detail Preservation: Maintains fine facial features in segmentation masks
• Consistency: Stable segmentation quality across different input conditions
• Efficiency: FP4 quantization achieves 4x memory savings with minimal quality loss

Deployment Options

• High-Quality Mode: BF16 precision for maximum accuracy
• Efficient Mode: FP4 quantization for resource-constrained environments
• Real-time Applications: Optimized inference pipeline for low-latency requirements This model represents a practical solution for face segmentation tasks, offering an excellent balance between accuracy, efficiency, and deployability across various hardware configurations

Example:

Control Images

Edited Image with Qwen-Image-Edit by promot `change the face to face segmentation mask`

After Lora Finetune with same prompt

Code

Lora Finetune of Qwen-Image-Edit Code here: https://github.com/tsiendragon/qwen-image-finetune

Download model

Download them in the Files & versions tab.