Model Card for CrystaLLM-pi_ft_alex_mp_20-text

Model Details

Model Description

CrystaLLM-pi_ft_alex_mp_20-text is an unconditional generative model for crystal structures. It is a GPT-2 decoder-only model that generates Crystallographic Information Files (CIFs) from the patterns learned during training, with no property or diffraction conditioning attached.

It was pretrained on LeMat-Bulk and finetuned on Alex_mp_20 CIF text. Within the CrystaLLM-pi model catalog it serves two roles:

1. Unconditional generation of plausible inorganic crystal structures, optionally steered by a composition or space group in the prompt.
2. The recommended base checkpoint for new conditional finetunes. Starting from it preserves an existing understanding of CIF syntax, composition, symmetry and coordinates, so a finetune only has to learn the property-to-structure relationship.

• Developed by: Bone et al. (University College London)
• Model type: Autoregressive Transformer, unconditional (~25.9M parameters)
• Language(s): CIF (Crystallographic Information File) syntax
• License: MIT
• Finetuned from model: c-bone/CrystaLLM-pi_base, the LeMat-Bulk pretrained base

Model Sources

• Repository: GitHub: CrystaLLM-pi (generation and finetuning)
• Dataset: HuggingFace: c-bone/alex_mp_20

Uses

Direct Use

Unconditional or prompt-steered generation of inorganic crystal structures, and as the starting checkpoint for property-conditioned finetuning with the Prefix or Residual architectures.

Out-of-Scope Use

• Property targeting: the model has no conditioning channel. Use a conditional model (or finetune this one) to request a property value.
• Disordered Systems: No native handling of partial occupancies or significant disorder.
• Organic/MOFs: Training data is inorganic, so organic frameworks are out of distribution.
• Extremely Large Unit Cells: the context window is 1024 tokens, so large cells will not fit.

Bias, Risks, and Limitations

• Distribution matching: output closely reflects the Alex_mp_20 distribution, so novelty is limited by design. Raise the generation temperature to move away from it.
• Validity is not stability: generated CIFs passing structural checks are not necessarily thermodynamically stable. Screen with an energy model before drawing conclusions.
• Computed data: training structures are relaxed DFT results rather than experimentally determined ones.

Getting started

For generation, use T2_load_and_generate.ipynb in CrystaLLM-pi. The training config is ft-alex-mp-20-text.jsonc.

Citation

@misc{bone2025discoveryrecoverycrystallinematerials,
      title={Discovery and recovery of crystalline materials with property-conditioned transformers},
      author={Cyprien Bone and Matthew Walker and Bradley A. A. Martin and Kuangdai Leng and Luis M. Antunes and Ricardo Grau-Crespo and Amil Aligayev and Javier Dominguez and Keith T. Butler},
      year={2025},
      eprint={2511.21299},
      archivePrefix={arXiv},
      primaryClass={cond-mat.mtrl-sci},
      url={https://arxiv.org/abs/2511.21299},
}