Activity Generation Model v0.2

mjpsm/activity-generation-model-v0.2 is a fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct designed to generate a simple next learning activity from a student's previous knowledge submission.

The model is part of an activity-generation workflow for the MyVillage Project. Given a knowledge submission describing something a student has learned, practiced, created, researched, or experienced, the model proposes one logical next activity that builds on that submission.

Model Input

The model accepts a student's previous knowledge submission.

Example:

updated project management doc for aurora phase 1 now includes srujana's task tracking template

Model Output

The model is trained to produce an activity containing:

• title
• description
• instructions
• activityType

The inference helper used during development returns the result in the following wrapper:

{
  "input": {
    "knowledge_submission": "..."
  },
  "output": {
    "title": "...",
    "description": "...",
    "instructions": "...",
    "activityType": "CREATE"
  }
}

The generated activityType must be one of seven supported activity categories:

Activity Type Purpose

---

REFLECTION Think or write about prior learning RESEARCH Gather additional information COLLABORATE Work with other people CREATE Produce a small artifact PRACTICE Build or reinforce a skill EXPERIENCE Attend, observe, or participate TEACH Explain or share knowledge with someone else

Activity Design

Version 0.2 was trained around activities intended to be small, actionable next steps rather than large assignments.

Activities are intended to:

• directly build on the student's previous knowledge submission;
• be promptly completable;
• provide clear and compact instructions;
• avoid unnecessarily large assignments such as multi-page reports, full applications, complete websites, or major projects;
• select an activity type that fits the generated activity.

Changes from v0.1

Version 0.2 represents a revised training objective and dataset.

The primary output contract is:

knowledge_submission
        ↓
activity-generation-model-v0.2
        ↓
title
description
instructions
activityType

estimatedMinutes, which was part of the earlier activity-generation target, was intentionally removed from the v0.2 output.

Keeping v0.2 separate from v0.1 preserves the earlier model as a baseline and makes it possible to compare versions without overwriting the previous model.

Training

The model was fine-tuned using supervised fine-tuning with LoRA.

Known training configuration:

Setting Value

---

Base model Qwen/Qwen2.5-0.5B-Instruct Fine-tuning method LoRA / supervised fine-tuning Epochs 5 Learning rate 2e-4 LR scheduler Cosine Warmup steps 5 Weight decay 0.01 Per-device training batch size 2 Gradient accumulation steps 8 Maximum sequence length 1024 Evaluation strategy Every epoch Checkpoint strategy Every epoch Best-model metric Validation loss load_best_model_at_end True

The training configuration selected the checkpoint with the lowest validation loss, rather than automatically using the final epoch.

Observed Training Results

The following metrics were observed during the v0.2 training run:

---

       Epoch  Training Loss     Validation        Entropy     Mean Token
                                      Loss                      Accuracy

---

           1       0.881803       0.830671       0.849002       0.796495

           2       0.738581   **0.792472**       0.747979       0.801279

           3       0.655374       0.794525       0.672648       0.801776

           4       0.605030       0.808197       0.643972       0.800806

---

Among the results recorded in this development session, Epoch 2 had the lowest observed validation loss (0.792472). The training setup used load_best_model_at_end=True and metric_for_best_model="eval_loss".

These token-level metrics should not be interpreted as an overall percentage of pedagogically correct activities. Generation quality, structural validity, relevance, and activity-type selection should be evaluated separately.

Example Generation

One observed v0.2 generation during development used this input:

updated project management doc for aurora phase 1 now includes srujana's task tracking template

and returned:

{
  "input": {
    "knowledge_submission": "updated project management doc for aurora phase 1 now includes srujana's task tracking template"
  },
  "output": {
    "title": "Task Tracking Template Refinement",
    "description": "Refine Srujana's task tracking template to improve accuracy.",
    "instructions": "Open the updated project management doc for Aurora Phase 1. Review the existing task tracking template. Identify any areas where you think it could be improved. Write down your suggestions in a short note (1-2 paragraphs) and suggest changes.",
    "activityType": "CREATE"
  }
}

This example is included to demonstrate the model's output structure. It should not be treated as a benchmark result.

Python Usage

Install the required packages:

pip install torch transformers accelerate

Then load the model with Transformers:

import json
import re
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "mjpsm/activity-generation-model-v0.2"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype="auto",
    device_map="auto",
)

model.eval()

SYSTEM_PROMPT = """You are an educational activity generator.

Given a student's previous knowledge submission, generate exactly one
simple next learning activity that directly builds on what the student
demonstrated.

Return valid JSON only with exactly these fields:
title, description, instructions, activityType.

activityType must be exactly one of:
REFLECTION, RESEARCH, COLLABORATE, CREATE, PRACTICE, EXPERIENCE, TEACH.
"""


def generate_activity(knowledge_submission: str):
    messages = [
        {
            "role": "system",
            "content": SYSTEM_PROMPT,
        },
        {
            "role": "user",
            "content": (
                "Knowledge submission:\n"
                f"{knowledge_submission}\n\n"
                "Generate the next activity."
            ),
        },
    ]

    prompt = tokenizer.apply_chat_template(
        messages,
        tokenize=False,
        add_generation_prompt=True,
    )

    inputs = tokenizer(
        prompt,
        return_tensors="pt",
    ).to(model.device)

    with torch.no_grad():
        generated = model.generate(
            **inputs,
            max_new_tokens=300,
            do_sample=False,
            pad_token_id=tokenizer.eos_token_id,
        )

    new_tokens = generated[0][inputs["input_ids"].shape[1]:]

    text = tokenizer.decode(
        new_tokens,
        skip_special_tokens=True,
    ).strip()

    # Extract the first JSON object from the generated response.
    match = re.search(r"\{.*\}", text, re.DOTALL)

    if not match:
        raise ValueError(
            f"Model did not return a JSON object. Raw output: {text}"
        )

    activity = json.loads(match.group(0))

    required_fields = {
        "title",
        "description",
        "instructions",
        "activityType",
    }

    missing = required_fields - set(activity.keys())

    if missing:
        raise ValueError(
            f"Generated activity is missing fields: {sorted(missing)}"
        )

    return activity


activity = generate_activity(
    "I practiced Python functions and learned how parameters and return values work."
)

print(json.dumps(activity, indent=2))

A typical response is expected to follow this shape:

{
  "title": "Practice Reusable Functions",
  "description": "Apply your understanding of parameters and return values in a small exercise.",
  "instructions": "Write three short Python functions that accept inputs and return a result. Test each function with at least two different inputs.",
  "activityType": "PRACTICE"
}

Generated wording and activity type can vary by input.

Intended Use

This model is intended for educational activity generation where a previous student knowledge submission is available and a small next learning step needs to be proposed.

Potential use cases include:

• generating a follow-up activity after a knowledge submission;
• supporting adaptive learning workflows;
• proposing a next step that extends prior learning;
• generating structured activity data for downstream application logic.

Evaluation Status

A held-out generation evaluation was being developed for v0.2. During that process, an initial evaluator incorrectly treated the model's nested { "input": ..., "output": ... } return structure as though the activity fields were at the top level. Those resulting predicted=None values were evaluator errors and are not reported here as model accuracy results.

A complete held-out generation benchmark---including exact activity-type accuracy and per-class accuracy---has not been established in this model card. It should be added after the corrected evaluator is run successfully.

Limitations

• Training examples include synthetically generated activities.
• Token-level accuracy does not measure whether an activity is educationally optimal.
• The generated activityType may not always be the only reasonable category for a given activity.
• Generated activities should be evaluated for relevance and appropriateness before being used in higher-stakes educational settings.
• The model may occasionally generate activities that are more involved than the intended small-task format.
• The model is designed to generate one next activity rather than a long-term learning plan.
• Performance on knowledge submissions substantially outside the training distribution has not yet been fully characterized.

Version History

v0.2

• Revamped activity-generation dataset
• Output simplified to four fields
• Removed estimatedMinutes
• Seven supported activity types
• Small next-step activity design
• Best-checkpoint selection based on validation loss

v0.1

Initial activity-generation model.