Bayaan - Advanced Quran Tafseer Search with AI Vector Models
π Overview
Bayaan is an AI-powered Quran Tafseer search system that uses multiple machine learning models to find relevant Islamic interpretations from 219,000 records across 84 scholarly books. It automatically picks the best AI approach for your query - simple keywords use TF-IDF, contextual searches use Word2Vec/BERT, making it like having an intelligent Islamic library at your fingertips.
π οΈ Tech Stack
- Flask - REST API framework
- scikit-learn - TF-IDF, cosine similarity
- SentenceTransformers - Omartificial-Intelligence-Space/Arabic-Triplet-Matryoshka-V2 for semantic search
- BERT/Word2Vec - Semantic embeddings
- pandas/numpy - Data processing
- Dataset: 219K Tafseer records from Altafsir.com
ποΈ The Dataset: A Treasure Trove of Islamic Knowledge
Bayaan is powered by the comprehensive Quran-Tafseer dataset from Hugging Face, created by MohamedRashad. This dataset is a goldmine for anyone interested in Islamic studies, natural language processing, or understanding the Quran's deeper meanings.
Dataset Highlights:
- π 84 Different Tafseer Books - From classical to contemporary scholars
- π 219,000 Rows of rich interpretative content
- π Source: All data collected from Altafsir.com
- π€ Language: Arabic (with English query support through AI)
What's Inside:
| Column | Description | Example |
|---|---|---|
surah_name |
Name of the Quran chapter | "Al-Fatiha", "Al-Baqarah" |
revelation_type |
Where the Surah was revealed | "Meccan" or "Medinan" |
ayah |
The specific Quranic verse | "Ψ¨ΩΨ³ΩΩ Ω Ψ§ΩΩΩΩΩΩ Ψ§ΩΨ±ΩΩΨΩΩ ΩΩ°ΩΩ Ψ§ΩΨ±ΩΩΨΩΩΩ Ω" |
tafsir_book |
Source of the interpretation | "Ibn Kathir", "Al-Jalalayn" |
tafsir_content |
The actual scholarly commentary | Detailed Arabic interpretation |
π€ How Bayaan Makes It Smart
Bayaan doesn't just do keyword matching - it understands context, meaning, and relationships between concepts using multiple AI approaches:
π Key Features
π€ Multi-Model AI Search
- TF-IDF Vectorization: Optimized for short queries (β€2 words)
- Word2Vec Embeddings: Perfect for medium-length queries (β€10 words)
- BERT Transformers: Advanced semantic understanding for long queries (>10 words)
- SentenceTransformers: State-of-the-art Arabic language model for advanced search
π― Intelligent Query Routing
- Hybrid Search Algorithm: Automatically selects the best AI model based on query characteristics
- Fallback Mechanisms: Ensures reliable results even when specific models are unavailable
- Contextual Understanding: Semantic similarity matching beyond keyword matching
π Advanced Search Capabilities
- Semantic Search: Find conceptually similar content, not just keyword matches
- Multi-field Search: Search across Ayahs, Tafseer content, Surah names, and more
- Similarity Scoring: Ranked results with confidence scores
- Flexible Result Limits: Configurable result counts (1-50 results)
ποΈ System Architecture
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β Query Input βββββΆβ Hybrid Router βββββΆβ AI Models β
βββββββββββββββββββ ββββββββββββββββββββ βββββββββββββββββββ
β β
βΌ βΌ
ββββββββββββββββββββ βββββββββββββββββββββββ
β Query Analysis β β β’ TF-IDF Matrix β
β - Length Check β β β’ Word2Vec Vectors β
β - Complexity β β β’ BERT Embeddings β
β - Language β β β’ SentenceTransformβ
ββββββββββββββββββββ βββββββββββββββββββββββ
β β
βΌ βΌ
ββββββββββββββββββββ βββββββββββββββββββββββ
β Result Ranking ββββββ Similarity Engine β
β - Cosine Sim β β - Vector Matching β
β - Score Fusion β β - Context Analysis β
ββββββββββββββββββββ βββββββββββββββββββββββ
π Required Data Files
| File | Description | Required | Size |
|---|---|---|---|
tafseer.csv |
Main Tafseer dataset | β Yes | Variable |
w2v_vectors.npy |
Pre-computed Word2Vec embeddings | β οΈ Optional | ~100MB |
bert_vectors.npy |
Pre-computed BERT embeddings | β οΈ Optional | ~200MB |
tafsir_embeddings.npy |
SentenceTransformer embeddings | β οΈ Optional | ~300MB |