Architecture: SNN (Spiking Neural Network) decoder with PonderNet-style adaptive K time-step routing. Custom model, not a transformer variant.
This is an in-training snapshot; not a finished model.
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"Brain2nd/NeuronSpark-V3-1.1B-Pretrain",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(
"Brain2nd/NeuronSpark-V3-1.1B-Pretrain"
)
The deepspeed/ directory contains 8-rank ZeRO-2 sharded optimizer state.
Use with DeepSpeed launcher to continue training from this exact point:
git lfs install
git clone https://huggingface.co/Brain2nd/NeuronSpark-V3-1.1B-Pretrain ./ckpt
deepspeed --num_gpus=8 train_pretrain.py \
--deepspeed_config ds_config.json \
--resume ./ckpt
model.safetensors |
HF-format bf16 weights |
config.json / generation_config.json |
model config |
tokenizer.json / tokenizer_config.json |
tokenizer |
modeling_neuronspark.py / configuration_neuronspark.py / __init__.py |
custom architecture (trust_remote_code=True) |
deepspeed/ |
ZeRO-2 optimizer state (8 ranks, fp32 master + Adam moments) |
training_state.pth |
step / tokens_seen / epoch metadata |
zero_to_fp32.py |
DeepSpeed helper to consolidate sharded state |
deepspeed/mp_rank_00_model_states.pt 2.5 GBmodel.safetensors 2.5 GBdeepspeed/zero_pp_rank_1_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_0_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_7_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_6_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_5_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_3_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_2_mp_rank_00_optim_states.pt 1.9 GBdeepspeed/zero_pp_rank_4_mp_rank_00_optim_states.pt 1.9 GBtraining_state.pth 1.4 kBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo Brain2nd/NeuronSpark-V3-1.1B-Pretrain ./model-folder