Run Gqa Rope

Custom PyTorch Transformer checkpoint trained on MeetingBank for meeting summarization research. This repository is part of the transformer-lab collection.

Model Details

Architecture

Static architecture diagram generated from this run's config.json, including model width, depth, sequence dimensions, and attention-specific settings.

Training Loss

Raw curve data is available in loss_curve.csv.

Available Models

Files

Usage

These checkpoints are from a custom PyTorch codebase, not a transformers.AutoModel checkpoint. Use the repo-native builder to instantiate the architecture, then load the checkpoint state dict.

from pathlib import Path

import torch
from huggingface_hub import hf_hub_download
from omegaconf import OmegaConf

import src  # registers components
from src.model.builder import build_transformer

repo_id = "Pradheep1647/run_gqa_rope-meetingbank-bs8-e20-fp32-19"

config_path = hf_hub_download(repo_id=repo_id, filename="config.json")
checkpoint_path = hf_hub_download(repo_id=repo_id, filename="meeting_model19.pt")

cfg = OmegaConf.load(config_path)
model = build_transformer(cfg)

state = torch.load(checkpoint_path, map_location="cpu")
model.load_state_dict(state["model_state_dict"])
model.eval()

print(f"Loaded {repo_id} from {Path(checkpoint_path).name}")

Notes

• This is a research checkpoint for comparing attention variants under the same MeetingBank setup.
• The config and tokenizers are included so future runs can reproduce the architecture and preprocessing assumptions.
• Use config.json as the source of truth for architecture parameters.