Data Engineering Mentor v2

An experimental English-language study assistant for data engineering topics. It was fine-tuned on 500 prompt/answer rows created from 50 original, source-linked concepts. Ten question phrasings per concept broaden prompt coverage while answers and references are retained. This is not 500 independent facts.

Use

from transformers import AutoModelForCausalLM, AutoTokenizer

repo = "LoNebula/data-engineering-mentor-v2"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
messages = [
    {"role": "system", "content": "You are a careful data engineering study assistant. State uncertainty."},
    {"role": "user", "content": "How should I make a batch pipeline idempotent?"},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
                                        return_tensors="pt")
generated = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(generated[0][inputs.shape[1]:], skip_special_tokens=True))

Limitations and references

This small model is a learning prototype, not a production runbook, benchmark, or authoritative reference. Validate advice against current vendor documentation and your system constraints. Its dataset contains 50 original concepts. The linked documentation is further reading and may not directly substantiate every recommendation; some answers are editorial synthesis, not claim-by-claim sourced assertions. The dataset card provides provenance, reference links, and license information.

License

The fine-tuned weights are published under Apache-2.0, subject to the base model's terms. Dataset licensing and attribution are documented separately on the dataset card.