FastAPI Qwen 0.6B REST API

This repository contains a production-ready (though lightweight) wrapper for the Qwen/Qwen3-0.6B model using FastAPI and JWT authentication.

Features

  • FastAPI Backend: Efficient RESTful endpoints.
  • JWT Security: Token-based authentication for the /generate endpoint.
  • Thinking Toggle: Option to hide or show the model's internal reasoning/thinking blocks via regex filtering.
  • OpenAI Chat Format: Compatible with list-of-messages input.

Files

  • app.py: The core FastAPI server logic.
  • generate_jwt.py: Utility to create access tokens.
  • client.py: Demo script for performing inference.

Quick Start

  1. Install dependencies: pip install fastapi uvicorn transformers torch pyjwt python-dotenv requests.
  2. Set your SECRET_KEY in a .env file.
  3. Run python app.py.
  4. Use client.py to send requests.