fine-tuned-model-indonesian-2

fine-tuned-model-indonesian-2 is an enhanced version (v2) of the Indonesian Language Model based on Llama-3.1-8B, fine-tuned using Unsloth and Hugging Face's TRL library. This model offers improved conversational alignment, better handling of complex Indonesian sentence structures, and higher retrieval accuracy for Legal AI RAG (Retrieval-Augmented Generation) applications.

---

📌 Model Details

---

💡 Key Improvements & Features

• Refined Indonesian Vocabulary: Improved fine-tuning dataset to minimize typos, repeated n-grams, and garbled outputs in formal Indonesian context.
• Enhanced Legal & Document RAG: Tailored to parse statutory references, regulatory articles (PP/UU), and organizational policies with high fidelity.
• Efficient 4-Bit Deployment: Utilizes NF4 quantization for low VRAM consumption while preserving generation quality during real-time inference.

---

⚙️ Recommended Inference Parameters

To maintain optimal consistency and prevent language degradation, use the following generation parameters:

---

🚀 How to Use

1. Using Hugging Face transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "hendrimardani/fine-tuned-model-indonesian-2"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype=torch.bfloat16,
    load_in_4bit=True
)

messages = [
    {
        "role": "system", 
        "content": "Anda adalah asisten AI profesional yang membantu menjawab pertanyaan hukum dan analisis dokumen dalam Bahasa Indonesia secara akurat dan lugas."
    },
    {
        "role": "user", 
        "content": "Jelaskan prosedur pendaftaran perjanjian kerja waktu tertentu (PKWT) menurut peraturan ketenagakerjaan di Indonesia."
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    do_sample=True,
    temperature=0.3,
    repetition_penalty=1.15,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id if tokenizer.pad_token_id is not None else tokenizer.eos_token_id
)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True)
print(response)

---

2. Integration with LangChain RAG Pipeline

from transformers import pipeline
from langchain_community.llms import HuggingFacePipeline

text_generation_pipeline = pipeline(
    task="text-generation",
    model=model,
    tokenizer=tokenizer,
    max_new_tokens=1024,
    do_sample=True,
    temperature=0.3,
    repetition_penalty=1.15,
    return_full_text=False,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.pad_token_id if tokenizer.pad_token_id is not None else tokenizer.eos_token_id
)

llm = HuggingFacePipeline(pipeline=text_generation_pipeline)

---

📄 Model Output Format

Berdasarkan PP No. 35 Tahun 2021, pencatatan Perjanjian Kerja Waktu Tertentu (PKWT) dilakukan dengan ketentuan berikut:
1. Pengusaha wajib mencatatkan PKWT kepada kementerian yang menyelenggarakan urusan pemerintahan di bidang ketenagakerjaan.
2. Pencatatan dilakukan secara daring (online) paling lama 7 (tujuh) hari kerja sejak penandatanganan PKWT.
3. Apabila sistem pencatatan secara daring belum tersedia, pencatatan dilakukan secara tertulis di dinas ketenagakerjaan setempat.

---

⚠️ Disclaimer

This model is intended for informational assistance and document analysis. Generated content does not constitute formal legal advice. Users should verify any output against primary statutory legal sources.