CENO model finetuned for one epoch on Oryza sativa CDS sequences.
CENO is derived from NVIDIA's Nemotron-H (Apache-2.0). The custom Transformers
remote code in this repository (configuration_ceno.py, modeling_ceno.py) is a
rename of the upstream Nemotron-H implementation.
This repository includes custom Transformers remote code for CENOForCausalLM
and CENOCharLevelTokenizer. Load with trust_remote_code=True.
model.safetensors: model weightsconfig.json: model config with auto_mapgeneration_config.json: generation configconfiguration_ceno.py, modeling_ceno.py: custom model codeceno_tokenizer.py, tokenizer_config.json, special_tokens_map.json, vocab.json: tokenizer filestraining_metrics.json: finetuning metricsfrom transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "CladeTeam/CENO-rice-cds"
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
The model code depends on PyTorch and the Mamba/Triton stack used by Nemotron-H.
The bundled config sets use_mamba_kernels=false, using the pure-PyTorch Mamba
fallback so no mamba-ssm/causal-conv1d install is required.
Finetuned for 1 epoch on rice CDS with learning_rate=5e-5, effective_batch_size=64, bf16, max_length=8192.
{
"species": "rice",
"train_loss": 10.05208391170438,
"eval_loss": 1.21553373336792,
"learning_rate": 5e-05,
"epochs": 1,
"epoch_losses": [
{
"epoch": 0.9987473903966597,
"eval_loss": 1.21553373336792
},
{
"epoch": 0.9987473903966597,
"eval_loss": 1.21553373336792
}
],
"n_gpu": 8,
"effective_batch_size": 64
}
These models are released to reproduce HTT/polyQ sequence scoring experiments. The average log-likelihood scores reflect sequence-model likelihood, not biological fitness or pathogenicity.
This model and its bundled code are released under the Apache License 2.0, inheriting the license of the upstream Nemotron-H model code (Copyright 2024 AI21 Labs Ltd. and the HuggingFace Inc. team; Copyright (c) 2025 NVIDIA CORPORATION). Modifications for CENO by CladeTeam.
Model weights and token IDs are unchanged. Current source and usage instructions are maintained at CladeTeam/CENO. Use the updated model revision instead of historical diagnostic archives.
The current loader, tokenizer and cache fixes are included. This checkpoint keeps its existing inference contract; a base-model CP profile is not inferred from its name or substituted for unknown post-training provenance.
The tested HF stack is PyTorch 2.7.0a0 (NVIDIA 25.03), Transformers 4.48.3, mamba-ssm 2.2.4, causal-conv1d 1.2.2.post1 and FlashAttention 2.7.3. HF and vLLM use separate environments. Numerical correctness is not a guarantee of repetition-free or biologically functional generation.
CladeTeam/CENO-base-1b-preview