TinyLlama-1B-Sonar-formatting

This repository contains an adapter-based fine-tuning of TinyLlama/TinyLlama-1.1B-Chat-v1.0 for response formatting and structural consistency.

The model was trained to produce answers in a clearer, more organized, and more readable style inspired by a small set of reference completions. The primary objective of this adaptation is not to improve factual knowledge or reasoning depth, but to bias the base model toward more coherent presentation patterns, including cleaner sectioning, more stable answer structure, and better formatting discipline.

The fine-tuning data consists of 457 instruction-response examples. The dataset was constructed from a small set of manually collected reference answers and expanded with synthetic examples generated in a few-shot setup to reinforce the target formatting style. As a result, this model is best understood as a style-and-structure adapter rather than a general capability upgrade.

This model may be useful for researchers and practitioners interested in:

• lightweight stylistic adaptation of small language models,
• format alignment through parameter-efficient fine-tuning,
• studying how small synthetic datasets can shape output structure and presentation quality.

Because the training objective is narrowly focused on formatting behavior, users should not assume improved factual accuracy, broader domain knowledge, or stronger reasoning performance relative to the original base model.

Model Details

Model Description

This model is an adapter-based fine-tuned variant of TinyLlama/TinyLlama-1.1B-Chat-v1.0, designed to improve response formatting quality, structural consistency, and readability.

The adaptation targets stylistic behavior rather than core capability expansion. In particular, the model was tuned to produce outputs with clearer organization, more stable formatting patterns, and more coherent answer structure inspired by a small set of high-quality reference responses.

• Developed by: zxc0zxc0zxc
• Model type: Causal language model with adapter-based supervised fine-tuning for formatting/style alignment
• Language(s) (NLP): English
• License: Apache-2.0
• Finetuned from model: TinyLlama/TinyLlama-1.1B-Chat-v1.0

Training Data

The training data was designed to transfer a target answer style rather than to expand the base model’s factual knowledge.

Data Collection Process

The dataset was constructed in two stages:

1. Reference Set Collection
   A small seed set of approximately 20 high-quality reference answers was manually collected from Sonar (Perplexity). These examples were selected because they exhibited the target stylistic properties sought in this project: clear structure, readable segmentation, concise lead statements, stable formatting, and well-organized explanatory flow.

2. Synthetic Dataset Expansion
   To increase coverage of formatting patterns while preserving the overall style, additional examples were synthetically generated using gpt-5-nano in a few-shot setup. The manually collected reference answers were used as demonstrations to condition the synthetic generation process toward the desired answer composition and formatting behavior.

The resulting dataset contains 457 supervised instruction-response pairs focused primarily on formatting transfer and structural alignment.

Data Characteristics

The dataset emphasizes:

• answer structure and segmentation,
• formatting consistency,
• concise lead summaries,
• readable explanatory organization,
• transfer of a specific stylistic response pattern.

It does not aim to serve as a general-purpose knowledge corpus or a benchmark for factuality. Consequently, improvements should be interpreted mainly in terms of output presentation and structural quality rather than raw reasoning performance or domain coverage.

Synthetic Data Note

A substantial portion of the dataset was synthetically generated. While this enabled efficient style transfer from a limited seed set, synthetic expansion may also propagate stylistic biases, repetitive structural patterns, or imperfections present in the prompting pipeline.

Dataset Availability

The full training dataset is not included in this repository at the time of release.

It can be found here: https://huggingface.co/datasets/zxc0zxc0zxc/sonar-formatting

Few-shot prompt template used for synthetic data generation

Below are examples of the target answer style. Match the same formatting, structure, and answer composition.

### Example 1
User: Who is Snoop Dogg?
Assistant: **Snoop Dogg, born Calvin Cordozar Broadus Jr. on October 20, 1971, in Long Beach, California, is an American rapper, singer, songwriter, record producer, actor, and one of the most influential figures in West Coast gangsta rap and hip-hop culture.**

His nickname originated from his mother, who likened his appearance to the Snoopy character from Peanuts, and he was raised in a musical family with gospel influences, playing piano and singing in a Baptist church before starting rap in sixth grade.

### Early Life and Troubles
Snoop grew up in tough Long Beach neighborhoods, joined the Rollin' 20 Crips gang, and faced multiple arrests for drug possession and cocaine after high school, leading to frequent incarcerations. He began recording demos with cousins Nate Dogg and friends Warren G and Lil' ½ Dead as part of group 213, with one tape (*Over the Counter*) drawing label interest.

### Music Career Breakthrough
Discovered by Dr. Dre via Warren G, Snoop featured on "Deep Cover" (1992) and Dre's album *The Chronic*, then debuted solo with *Doggystyle* (1993) under Death Row Records, which topped the Billboard 200, sold over 800,000 copies in its first week, and spawned hits like "Who Am I (What's My Name)?" and "Gin and Juice." His second album, *Tha Doggfather* (1996), followed amid label turmoil, including friend Tupac Shakur's death and Suge Knight's issues.

### Later Career and Transitions
After leaving Death Row, Snoop signed with No Limit Records, releasing *Da Game Is to Be Sold, Not to Be Told* (1998), *No Limit Top Dogg* (1999, over 1.5 million copies), and *Tha Last Meal* (2000, over 2 million copies), shifting toward a smoother sound. He reformed 213 with Warren G and Nate Dogg for *The Hard Way* (2004) and has maintained a career spanning over three decades with a laid-back style, clever wordplay, and G-funk influence. Beyond music, he has acted, produced, and become a cultural icon.

### Example 2
User: Explain what a vector database is in simple terms.
Assistant: A **vector database** is a specialized database designed to store and search data as mathematical representations called **vectors** – essentially lists of numbers that capture the meaning of information.

Here's how it works in simple terms:

**The Basic Concept**

Traditional databases store data in tables with rows and columns, comparing items by exact matching. Vector databases instead organize information as points in a multi-dimensional space, allowing them to find similar items based on how close they are to each other. Think of it like plotting books on a map where similar books cluster together – you can find related books by looking at what's nearby rather than searching for exact titles.

**The Process**

When you add data to a vector database, machine learning models first convert it (text, images, audio, etc.) into vectors – fixed-length arrays of numbers that represent meaning. The database then stores these vectors and uses specialized algorithms to organize them so that similar vectors are grouped together. When you search, the database finds vectors nearest to your query vector and returns the associated original content.

**Why This Matters**

Vector databases enable **semantic search** – finding information based on meaning rather than keywords. This is crucial for AI applications like large language models, recommendation systems, and anomaly detection. Since the data is converted to vectors once and stored, queries return results in milliseconds rather than requiring the model to process everything from scratch.

**Key Features**

Vector databases provide practical capabilities like metadata filtering, data management, scalability, and security – making them suitable for real-world AI applications beyond just raw similarity search.

[Prompt truncated in the model card for brevity.]

Now answer the next user request in the same style.

Training

The model was fine-tuned in a Google Colab environment using GPU-backed training with a lightweight adapter-based setup suitable for low-resource experimentation.

Training was performed with the Hugging Face Trainer API under a small-batch regime optimized for limited hardware availability. The objective of training was supervised format/style adaptation: to bias the base model toward more consistent answer structure, clearer formatting, and improved response organization, rather than to increase factual knowledge or reasoning capability.

Training Configuration

training_args = TrainingArguments(
    output_dir=OUTPUT_DIR,
    per_device_train_batch_size=2,
    per_device_eval_batch_size=2,
    gradient_accumulation_steps=8,
    num_train_epochs=3,
    learning_rate=1e-4,
    weight_decay=0.01,
    logging_steps=10,
    eval_strategy="steps",
    eval_steps=20,
    save_strategy="steps",
    save_steps=20,
    save_total_limit=2,
    fp16=True,
    bf16=False,
    report_to="none",
    load_best_model_at_end=True,
    lr_scheduler_type="cosine",
    warmup_ratio=0.05,
    max_grad_norm=0.3,
    optim="paged_adamw_8bit",
)

Optimization Setup

The training run used:

• batch size per device: 2
• evaluation batch size per device: 2
• gradient accumulation steps: 8
• effective train batch size: 16
• number of epochs: 3
• learning rate: 1e-4
• weight decay: 0.01
• scheduler: cosine
• warmup ratio: 0.05
• max gradient norm: 0.3
• optimizer: paged_adamw_8bit
• mixed precision: FP16

This configuration was chosen to make fine-tuning feasible on constrained hardware while preserving stable optimization dynamics for a small supervised dataset.

Training Result Summary

The final training run reported the following metrics: global steps: 58

• training loss: 1.8804
• train runtime: 870.64 seconds
• train samples / second: 1.05
• train steps / second: 0.067
• total FLOPs: 5.88e15
• final reported epoch: 2.0

Notes

Because this project is focused on formatting adaptation, the most meaningful evaluation criterion is qualitative output structure rather than training loss alone. In this setting, the model is intended to be assessed primarily on improvements in readability, segmentation, structural consistency, and adherence to the target answer style.

Examples

Limitations

This model is a lightweight formatting-oriented adapter and should be understood within that scope.

Scope of Improvement

The fine-tuning objective was primarily stylistic: to improve answer structure, readability, and formatting consistency. It was not designed to substantially improve:

• factual accuracy,
• reasoning depth,
• domain expertise,
• long-context reliability,
• instruction-following robustness outside the formatting domain.

As a result, any gains should be interpreted mainly as improvements in presentation quality rather than core language-model capability.

Dataset Size and Composition

The training dataset is relatively small (457 examples) and was constructed specifically for style transfer. This limits the breadth of behaviors the model can reliably generalize to. In particular, the model may perform best on prompts that resemble the structural patterns represented in the training data and less consistently on substantially different prompt types.

In addition, a substantial portion of the dataset was synthetically generated through few-shot prompting. While this was useful for amplifying the target style, it may also introduce:

• repetitive structural tendencies,
• synthetic artifacts,
• hidden prompt biases,
• over-regularization toward a narrow answer template.

Narrow Objective

This adaptation is intended to bias the model toward a specific response style inspired by the reference examples. Consequently, it may sometimes prefer stylistic regularity over flexibility, brevity, or task-specific adaptation. In some cases, this can lead to outputs that are overly structured for simple prompts.

No Claim of Benchmark Improvement

This repository does not claim broad performance improvements over the base model on standard reasoning, factuality, or knowledge benchmarks. The model should therefore be evaluated primarily through qualitative inspection of formatting behavior and response organization.

Inheritance from the Base Model

Because this is an adapter-based fine-tuning of TinyLlama/TinyLlama-1.1B-Chat-v1.0, the model retains the general limitations of the underlying base model, including possible hallucinations, factual errors, sensitivity to prompt phrasing, and uneven performance on complex or specialized tasks.

Appropriate Use

This model is best suited for experimentation with:

• formatting/style transfer,
• lightweight supervised adaptation,
• structured answer presentation.

It should not be treated as a reliability-focused, domain-specialized, or safety-critical system without additional evaluation.