Base CENO language model. This is a preview release.
CENO is derived from NVIDIA's Nemotron-H (Apache-2.0). The custom Transformers
remote code in this repository (configuration_ceno.py, modeling_ceno.py) is a
rename of the upstream Nemotron-H implementation; the model weights are unchanged.
This repository includes custom Transformers remote code for CENOForCausalLM
and CENOCharLevelTokenizer. Load with trust_remote_code=True.
model.safetensors: model weightsconfig.json: model config with auto_mapgeneration_config.json: generation configconfiguration_ceno.py, modeling_ceno.py: custom model codeceno_tokenizer.py, tokenizer_config.json, special_tokens_map.json, vocab.json: tokenizer filesfrom transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "CladeTeam/CENO-base-1b-preview"
model = AutoModelForCausalLM.from_pretrained(repo_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(repo_id, trust_remote_code=True)
The model code depends on PyTorch and the Mamba/Triton stack used by Nemotron-H.
The bundled config sets use_mamba_kernels=true; if mamba-ssm and
causal-conv1d are not installed, set config.use_mamba_kernels=False before
loading to fall back to the pure-PyTorch Mamba path.
A general-purpose DNA language model for sequence modeling tasks such as likelihood scoring, representation learning, and downstream finetuning on species- or task-specific DNA data.
This model and its bundled code are released under the Apache License 2.0, inheriting the license of the upstream Nemotron-H model code (Copyright 2024 AI21 Labs Ltd. and the HuggingFace Inc. team; Copyright (c) 2025 NVIDIA CORPORATION). Modifications for CENO by CladeTeam.
Model weights and token IDs are unchanged. Current source and usage instructions are maintained at CladeTeam/CENO. Use the updated model revision instead of historical diagnostic archives.
The current loader, tokenizer and cache fixes are included. This checkpoint keeps its existing inference contract; a base-model CP profile is not inferred from its name or substituted for unknown post-training provenance.
The tested HF stack is PyTorch 2.7.0a0 (NVIDIA 25.03), Transformers 4.48.3, mamba-ssm 2.2.4, causal-conv1d 1.2.2.post1 and FlashAttention 2.7.3. HF and vLLM use separate environments. Numerical correctness is not a guarantee of repetition-free or biologically functional generation.