🧱 Rebar — heading classifier

Rebar restores the heading hierarchy of Japanese documents from flattened plain text (第1章 → 1. → (1) → ア), then chunks along that hierarchy so retrieval keeps articles, tables and FAQs intact. This repository hosts the heading-line detector — the one learned component. It is deliberately tiny: a logistic regression over 13 engineered features, sigmoid(x·w + b), exported to ONNX (~0.6 KB) and GGUF (~0.7 KB).

• 👉 Code & docs: https://github.com/NagaYu/rebar
• 👉 Live demo (Space): https://huggingface.co/spaces/NagaYu/rebar
• 👉 Dataset: https://huggingface.co/datasets/NagaYu/rebar-structure

Rebar is not a large multimodal PDF parser and does no layout analysis. It is the stage after text extraction. Its edge is local, cheap, layout-independent.

Why a tiny model works

The hard part isn't a big network — it's the document-level inference around it. The same marker means different depths in different documents (1. is top-level in a memo, second-level under 第3章), so depth is inferred jointly for the whole document from its nesting evidence, using the Japanese public-document / statutory chains as a prior. The classifier here only answers "is this line a heading?"; scheme inference, tree decoding and chunking are deterministic algorithms in the package.

Results (synthetic benchmark)

Downstream, chunking along the restored tree drops known failure modes to 0% (article↔proviso, table↔title, FAQ Q↔A) and lifts complete-answer recall@1 (e.g. tables 96.6% vs. ~64% for fixed-length).

Usage

Highest level — the full pipeline (recommended):

pip install "git+https://github.com/NagaYu/rebar"

from rebar import Rebar
text = open("doc.txt", encoding="utf-8").read()
print(Rebar().outline(text).outline())      # restored hierarchy
chunks = Rebar().chunk(text, max_len=800)    # structure-aware chunks

Just the ONNX classifier (line → P(heading)):

import numpy as np, onnxruntime as ort
from rebar.model import line_features           # 13-dim feature extractor
sess = ort.InferenceSession("rebar_heading_clf.onnx")
x = line_features("第3条 目的").astype(np.float32)[None, :]
print(sess.run(None, {"features": x})[0])       # ~1.0 => heading

Files

Features (13)

has_marker, marker_rank_norm, is_appendix, is_short, title_len_norm, ends_sentence, has_tab, looks_page_number, digit_ratio, punct_ratio, spaced_ratio, layout_indent, layout_heading_font. The two layout_* slots are optional auxiliary inputs — zero when no layout is available (Rebar runs without layout).

License

Apache-2.0. Training data is the CC0 synthetic corpus in NagaYu/rebar-structure.