AMRBART model is continually pre-trained on the English text and AMR Graphs based on the BART model. It was introduced in the paper: Graph Pre-training for AMR Parsing and Generation by bai et al. in ACL 2022 and first released in this repository.
AMRBART follows the BART model which uses a transformer encoder-encoder architecture. AMRBART is pre-trained with 6 tasks:
AMRBART is particularly effective when fine-tuned for AMR parsing and AMR-to-text generation tasks.
The AMRBART model is pre-trained on AMR3.0, a dataset consisting of 55,635 training instances and English Gigaword (we randomly sampled 200,000 sentences).
You can use the raw model for either AMR encoding or AMR parsing, but it's mostly intended to be fine-tuned on a downstream task.
Here is how to initialize this model in PyTorch:
from transformers import BartForConditionalGeneration
model = BartForConditionalGeneration.from_pretrained("xfbai/AMRBART-base")
Please refer to this repository for tokenizer initialization and data preprocessing.
Please cite this paper if you find this model helpful
@inproceedings{bai-etal-2022-graph,
title = "Graph Pre-training for {AMR} Parsing and Generation",
author = "Bai, Xuefeng and
Chen, Yulong and
Zhang, Yue",
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = may,
year = "2022",
address = "Online",
publisher = "Association for Computational Linguistics",
url = "todo",
doi = "todo",
pages = "todo"
}
pytorch_model.bin 569 MBmodel.safetensors 569 MB