Hallutok-SpaceLLM
Domain-Specialized Language Model for Space Systems, Mission Knowledge & Aerospace Reasoning
Hallutok-SpaceLLM is a parameter-efficient fine-tune of Llama-3.2-3B-Instruct, specifically adapted for the space and aerospace domain. It is trained on a carefully curated mixture of real-world data covering NASA, SpaceX, and ISRO missions, space systems engineering concepts, and related technical knowledge.
The model aims to deliver higher factual density, better terminology alignment, and improved response quality on space-related queries compared to general-purpose models of similar size.
Model Overview
| Property | Value |
|---|---|
| Model Name | Hallutok-SpaceLLM |
| Base Model | unsloth/Llama-3.2-3B-Instruct |
| Architecture | Llama 3.2 (3 Billion Parameters) |
| Fine-tuning Method | QLoRA (4-bit) + Unsloth |
LoRA Rank (r) |
16 |
| LoRA Alpha | 16 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Max Sequence Length | 2048 |
| Training Precision | 4-bit NF4 → Merged 16-bit |
| Training Framework | Unsloth + TRL + Hugging Face Transformers |
| Hardware | NVIDIA T4 (Google Colab Free Tier) |
| Training Steps | 150 |
| License | Apache 2.0 |
Key Capabilities
- Strong factual knowledge of NASA, SpaceX, and ISRO missions
- Understanding of launch vehicles (Falcon 9, PSLV, GSLV, Starship, etc.)
- Space systems and mission engineering concepts
- Ability to explain complex space topics in structured and accessible language
- Lightweight enough to run on consumer GPUs and free-tier environments
Training Data
The model was fine-tuned on a mixture of the following high-signal sources:
- AstroChat — High-quality conversational dataset focused on space mission engineering and astronautics
- SpaceX Launch History — Real mission data converted into natural language instruction format
- ISRO Mission Records — Structured mission information transformed into instruction-response pairs
- NASA Exoplanet Archive (selected subset)
- Deep Space Probes dataset (Voyager, Cassini, and related missions)
Structured datasets were carefully converted into clean instruction-response format before training.
Evaluation
A domain-specific evaluation was conducted using a curated set of factual questions covering major NASA, SpaceX, and ISRO topics.
| Metric | Result |
|---|---|
| Evaluation Set Size | 8 core questions |
| Keyword / Factual Match Accuracy | Moderate to High |
| Qualitative Performance | Strong on mission facts, vehicle names, and agency programs |
Note: This is an initial domain evaluation. A more comprehensive benchmark suite (including long-form technical questions and hallucination analysis) is planned for future releases.
Example Outputs
User: What is the Chandrayaan-3 mission?
Hallutok-SpaceLLM: Chandrayaan-3 is the third lunar exploration mission by the Indian Space Research Organisation (ISRO). It successfully achieved a soft landing near the Moon’s south pole in August 2023...
User: Who developed the Falcon 9 rocket?
Hallutok-SpaceLLM: The Falcon 9 was developed by SpaceX...
User: What was the main goal of NASA’s Apollo program?
Hallutok-SpaceLLM: The primary goal of NASA’s Apollo program was to land humans on the Moon and return them safely to Earth...
Intended Use
This model is suitable for:
- Space education and outreach applications
- Technical question-answering systems focused on aerospace
- Retrieval-Augmented Generation (RAG) pipelines in the space domain
- Lightweight domain-specific assistants
- Research and experimentation in domain-adaptive language models
Not recommended for safety-critical systems, real-time mission operations, or as a sole source of engineering truth without human verification.
How to Use
from unsloth import FastLanguageModel
from transformers import TextStreamer
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "joel-pawar/hallutok-SpaceLLM",
max_seq_length = 2048,
dtype = None,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
messages = [
{"role": "user", "content": "Explain the significance of Chandrayaan-3 landing near the lunar south pole."}
]
inputs = tokenizer.apply_chat_template(
messages,
tokenize = True,
add_generation_prompt = True,
return_tensors = "pt"
).to("cuda")
streamer = TextStreamer(tokenizer)
_ = model.generate(
input_ids = inputs,
streamer = streamer,
max_new_tokens = 256,
temperature = 0.6,
do_sample = True,
)
Limitations
- Knowledge coverage is limited by the size and diversity of the fine-tuning dataset.
- The model may still hallucinate on highly specialized or low-resource technical topics.
- Not a substitute for official documentation from NASA, ISRO, or SpaceX.
- Best results are achieved when combined with retrieval systems for up-to-date information.
Technical Details
- Trained using Unsloth for significantly improved memory efficiency and training speed.
- Applied 4-bit quantization with gradient checkpointing to enable training on free-tier GPUs.
- LoRA adapters were applied to both attention and MLP layers for stronger domain adaptation.
- Final weights available in both LoRA adapter format and merged 16-bit format.
Citation
If you use this model in your research or applications, please cite:
@misc{hallutok-spacellm-2026,
title = {Hallutok-SpaceLLM: A Domain-Adapted Language Model for NASA, SpaceX, and ISRO Knowledge},
author = {Joel Pawar},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/joel-pawar/hallutok-SpaceLLM}}
}
Acknowledgements
- Base Model: Meta Llama 3.2 (via Unsloth)
- Training Stack: Unsloth, Hugging Face TRL, Transformers, PEFT
- Data Sources: Public datasets related to NASA, SpaceX, ISRO, and the broader space research community
Developed by: Joel Pawar
Model Type: Domain-Adaptive Causal Language Model
Version: 1.0